3604d63184
backport https://github.com/apache/doris/pull/40568 https://github.com/apache/doris/pull/40455 https://github.com/apache/doris/pull/40456 https://github.com/apache/doris/pull/40153 https://github.com/apache/doris/pull/34384 Test result: 2024-09-11 11:00:45.618 INFO [suite-thread-1] (SuiteContext.groovy:309) - Recover original connection 2024-09-11 11:00:45.619 INFO [suite-thread-1] (Suite.groovy:359) - Execute sql: REVOKE SELECT_PRIV ON test_partitions_schema_db.duplicate_table FROM partitions_user 2024-09-11 11:00:45.625 INFO [suite-thread-1] (SuiteContext.groovy:299) - Create new connection for user 'partitions_user' 2024-09-11 11:00:45.632 INFO [suite-thread-1] (Suite.groovy:1162) - Execute tag: select_check_5, sql: select TABLE_CATALOG,TABLE_SCHEMA,TABLE_NAME,PARTITION_NAME,SUBPARTITION_NAME,PARTITION_ORDINAL_POSITION,SUBPARTITION_ORDINAL_POSITION,PARTITION_METHOD,SUBPARTITION_METHOD,PARTITION_EXPRESSION,SUBPARTITION_EXPRESSION,PARTITION_DESCRIPTION,TABLE_ROWS,AVG_ROW_LENGTH,DATA_LENGTH,MAX_DATA_LENGTH,INDEX_LENGTH,DATA_FREE,CHECKSUM,PARTITION_COMMENT,NODEGROUP,TABLESPACE_NAME from information_schema.partitions where table_schema="test_partitions_schema_db" order by TABLE_CATALOG,TABLE_SCHEMA,TABLE_NAME,PARTITION_NAME,SUBPARTITION_NAME,PARTITION_ORDINAL_POSITION,SUBPARTITION_ORDINAL_POSITION,PARTITION_METHOD,SUBPARTITION_METHOD,PARTITION_EXPRESSION,SUBPARTITION_EXPRESSION,PARTITION_DESCRIPTION,TABLE_ROWS,AVG_ROW_LENGTH,DATA_LENGTH,MAX_DATA_LENGTH,INDEX_LENGTH,DATA_FREE,CHECKSUM,PARTITION_COMMENT,NODEGROUP,TABLESPACE_NAME 2024-09-11 11:00:45.644 INFO [suite-thread-1] (SuiteContext.groovy:309) - Recover original connection 2024-09-11 11:00:45.645 INFO [suite-thread-1] (ScriptContext.groovy:120) - Run test_partitions_schema in /root/doris/workspace/doris/regression-test/suites/query_p0/system/test_partitions_schema.groovy succeed 2024-09-11 11:00:45.652 INFO [main] (RegressionTest.groovy:259) - Start to run single scripts 2024-09-11 11:01:10.321 INFO [main] (RegressionTest.groovy:380) - Success suites: /root/doris/workspace/doris/regression-test/suites/query_p0/system/test_partitions_schema.groovy: group=default,p0, name=test_partitions_schema 2024-09-11 11:01:10.322 INFO [main] (RegressionTest.groovy:459) - All suites success. ____ _ ____ ____ _____ ____ | _ \ / \ / ___/ ___|| ____| _ \ | |_) / _ \ \___ \___ \| _| | | | | | __/ ___ \ ___) |__) | |___| |_| | |_| /_/ \_\____/____/|_____|____/ 2024-09-11 11:01:10.322 INFO [main] (RegressionTest.groovy:410) - Test 1 suites, failed 0 suites, fatal 0 scripts, skipped 0 scripts 2024-09-11 11:01:10.322 INFO [main] (RegressionTest.groovy:119) - Test finished 2024-09-11 11:03:00.712 INFO [suite-thread-1] (Suite.groovy:1162) - Execute tag: select_check_5, sql: select * from information_schema.table_options ORDER BY TABLE_CATALOG,TABLE_SCHEMA,TABLE_NAME,TABLE_MODEL,TABLE_MODEL_KEY,DISTRIBUTE_KEY,DISTRIBUTE_TYPE,BUCKETS_NUM,PARTITION_NUM; 2024-09-11 11:03:00.729 INFO [suite-thread-1] (SuiteContext.groovy:309) - Recover original connection 2024-09-11 11:03:00.731 INFO [suite-thread-1] (ScriptContext.groovy:120) - Run test_table_options in /root/doris/workspace/doris/regression-test/suites/query_p0/system/test_table_options.groovy succeed 2024-09-11 11:03:04.817 INFO [main] (RegressionTest.groovy:259) - Start to run single scripts 2024-09-11 11:03:28.741 INFO [main] (RegressionTest.groovy:380) - Success suites: /root/doris/workspace/doris/regression-test/suites/query_p0/system/test_table_options.groovy: group=default,p0, name=test_table_options 2024-09-11 11:03:28.742 INFO [main] (RegressionTest.groovy:459) - All suites success. ____ _ ____ ____ _____ ____ | _ \ / \ / ___/ ___|| ____| _ \ | |_) / _ \ \___ \___ \| _| | | | | | __/ ___ \ ___) |__) | |___| |_| | |_| /_/ \_\____/____/|_____|____/ 2024-09-11 11:03:28.742 INFO [main] (RegressionTest.groovy:410) - Test 1 suites, failed 0 suites, fatal 0 scripts, skipped 0 scripts 2024-09-11 11:03:28.742 INFO [main] (RegressionTest.groovy:119) - Test finished *************************** 7. row *************************** PartitionId: 18035 PartitionName: p100 VisibleVersion: 2 VisibleVersionTime: 2024-09-11 10:59:28 State: NORMAL PartitionKey: col_1 Range: [types: [INT]; keys: [83647]; ..types: [INT]; keys: [2147483647]; ) DistributionKey: pk Buckets: 10 ReplicationNum: 1 StorageMedium: HDD CooldownTime: 9999-12-31 15:59:59 RemoteStoragePolicy: LastConsistencyCheckTime: NULL DataSize: 2.872 KB IsInMemory: false ReplicaAllocation: tag.location.default: 1 IsMutable: true SyncWithBaseTables: true UnsyncTables: NULL CommittedVersion: 2 RowCount: 4 7 rows in set (0.01 sec) --------- Co-authored-by: Mingyu Chen <morningman.cmy@gmail.com>
Guide for test cases
General Case
-
Write "def" before variable names; otherwise, they will be global variables and may be affected by other cases running in parallel.
Problematic code:
ret = ***Correct code:
def ret = *** -
Avoid setting global session variables or modifying cluster configurations in cases, as it may affect other cases.
Problematic code:
sql """set global enable_pipeline_x_engine=true;"""Correct code:
sql """set enable_pipeline_x_engine=true;""" -
If it is necessary to set global variables or modify cluster configurations, specify the case to run in a nonConcurrent manner.
-
For cases involving time-related operations, it is best to use fixed time values instead of dynamic values like the
now()function to prevent cases from failing after some time.Problematic code:
sql """select count(*) from table where created < now();"""Correct code:
sql """select count(*) from table where created < '2023-11-13';""" -
After streamloading in a case, add a sync to ensure stability when executing in a multi-FE environment.
Problematic code:
streamLoad { ... } sql """select count(*) from table """Correct code:
streamLoad { ... } sql """sync""" sql """select count(*) from table """ -
For UDF cases, make sure to copy the corresponding JAR file to all BE machines.
-
Do not create the same table in different cases under the same directory to avoid conflicts.
Compatibility case
Refers to the resources or rules created on the initial cluster during FE testing or upgrade testing, which can still be used normally after the cluster restart or upgrade, such as permissions, UDF, etc.
These cases need to be split into two files, load.groovy and xxxx.groovy, placed in a folder, and tagged with the restart_fe group label, example.