Commit Graph

14711 Commits

Author SHA1 Message Date
3044b8397e [feature](fe) Add coverage tool for FE UT (#26203) 2023-11-11 19:54:04 +08:00
196fadc044 [enhancement](metrics) enhance visibility of flush thread pool (#26544) 2023-11-11 19:53:24 +08:00
8b33b0c4a4 [Fix](row store) cache invalidate key should not include sequence column (#26771) 2023-11-11 01:30:32 -06:00
ca47d75e83 [fix](regression) Add regression for group commit executed on observe… (#26692) 2023-11-10 18:53:45 +08:00
70fdd1f1af [fix](ci) fix bug, tpch pipeline upload log (#26627)
* [fix](ci) fix bug, tpch pipeline upload log
Co-authored-by: stephen <hello-stephen@qq.com>
2023-11-10 18:01:40 +08:00
fd43e64a72 [Enhancement](sql-cache) Use update time of hive to avoid cache miss through multi fe nodes. (#26424)
Now the update time of hms table is generated by every FE node (Use `System.currentTimestamp()` separately), so the update time of a hms table may be different between FE nodes, always the same query can not hit the sql-cache if we submit it more than one times through different FE nodes. This pr mainly do following changes to avoid this problem.

- Use the `transient_lastDdlTime` instead of `System.currentTimestamp` as the `schemaUpdateTime` of hms tables
- Use the `eventTime` in hms event instead of `System.currentTimestamp` as the update time when processing hms events
2023-11-10 17:36:00 +08:00
8ee237c55a [Enhance](regression)enhance case test_hdfs_json_load #26358
enhance case test_hdfs_json_load
2023-11-10 17:29:11 +08:00
0e0cd3b256 [fix](action) Update pr-approve-status.yml (#26577)
According to https://docs.github.com/en/rest/pulls/reviews?apiVersion=2022-11-28#list-reviews-for-a-pull-request,
the number of results per page default is 30 (max 100).
review of  APPROVED after 30 will not be listed,
change to 100 to fix it.
2023-11-10 17:01:37 +08:00
Pxl
2712bb9f60 [Bug](decimalv2) getCmpType return decimalv2 when lhs/rhs type both is decimalv2 (#26705)
getCmpType return decimalv2 when lhs/rhs type both is decimalv2
2023-11-10 16:21:28 +08:00
59efebce3b [opt](nereids) estimate join cost when col stats are not available (#26086)
no stats left zigzag
2023-11-10 16:13:53 +08:00
0749d632c4 [feature](diagnose) diagnose for cluster balance (#26085) 2023-11-10 15:31:58 +08:00
4ebb517af0 [fix](be-ut) Fix compilation errors caused by missing opentelemetry headers (#26739) 2023-11-10 14:58:46 +08:00
ce64f0c917 [enhancement](Nereids): add phase in shape string (#26682) 2023-11-10 14:56:28 +08:00
5c3fed216d [fix](transaction) Fix publish txn wait too long when not meet quorum (#26659) 2023-11-10 14:55:26 +08:00
9f6c6ffc92 [regression-test](stream load)Invalid file format check (#26713) 2023-11-10 14:53:01 +08:00
899630d0eb [chore](key_util) remove useless null_first parameter (#26635)
Doris always put null in the first when sorting key, the parameter null_first of encode_keys is useless.
2023-11-10 14:27:47 +08:00
cdba4936b4 [feature](nereids) Support group commit insert (#26075) 2023-11-10 14:20:14 +08:00
019fb956d3 [docs](cache) Refactor query-cache docs (#26418) 2023-11-10 13:57:20 +08:00
7878c08e15 [Revert](merge-on-write) Don't use delete bitmap to mark delete for rows with delete sign when sequence column doesn't exist (#26721) 2023-11-10 13:55:40 +08:00
27a21aa150 [fix](balance) Delete useless debug log (#26732) 2023-11-10 12:57:13 +08:00
Pxl
c02c00974f [Bug](agg-state) fix file load insert wrong data to agg_state (#26581)
fix file load insert wrong data to agg_state
2023-11-10 11:03:13 +08:00
7ce746654a [test](jdbc) add doris and sqlserver jdbc catalog test case (#26656) 2023-11-10 10:32:09 +08:00
76c0a906e5 [regression test](stream load) load_to_single_tablet (#26720) 2023-11-10 10:20:24 +08:00
7754791146 [improvement](disk balance) Prevent duplicate disk balance tasks afte… (#25990) 2023-11-10 10:14:42 +08:00
2bf48d7829 Revert "[Coverage](BE) Delete vinfo_func in BE (#26562)" (#26723)
This reverts commit 01094fd25ed539a8025066d8823c1e907109048a.
2023-11-10 10:14:11 +08:00
d767804815 [feature](merge-cloud) Decouple rowset id generator and local rowsets gc implementation (#25921) 2023-11-10 10:07:02 +08:00
49cffd0bc9 [fix](JdbcCatalog) fix that the predicate column name does not have back quote when querying the JDBC appearance (#26479) 2023-11-10 09:54:39 +08:00
328e5b120b [fix](group commit) Group commit support sequence column for unique table (#26652) 2023-11-09 22:43:37 +08:00
02fe71dcd6 [test](serialisation) Serialise some cases and enable str_to_date tests #26651
1 enable the cases about str_to_date, which have been muted because some parallel config influence.
2 serialise some cases which called admin set config
2023-11-09 21:53:32 +08:00
1722549381 [test](statistics)Add hive statistics all data type p0 test (#26676) 2023-11-09 21:48:18 +08:00
d988193d39 [pipelineX](shuffle) block exchange sink by memory usage (#26595) 2023-11-09 21:28:22 +08:00
0af8223894 [chore](ci) support to run regression pipeline mutiple times (#26701)
Co-authored-by: stephen <hello-stephen@qq.com>
2023-11-09 20:16:38 +08:00
8434389358 [fix](jdbc) fix clickhouse catalog arr nullable and add case (#26639) 2023-11-09 19:32:05 +08:00
30805a2d79 [fix](Nereids): skip OptimizeGroupExpressionJob if groupExpression is unused (#26641) 2023-11-09 19:21:04 +08:00
c07a70e22a [Fix](orc-reader) Add missing break introduced by #26548. (#26633)
Add missing break introduced by #26548. Sorry for this mistake.
2023-11-09 18:29:44 +08:00
fac1b2b192 [fix](planner)cast floating point type to bigint for bit functions (#26598) 2023-11-09 18:06:06 +08:00
a5565f68b2 [Refactor](opentelemetry) Remove opentelemetry (#26605) 2023-11-09 18:05:34 +08:00
eca747413d [Fix](partial update) Fix core when doing partial update on tables with row column after schema change (#26632) 2023-11-09 18:00:05 +08:00
8bb6606662 [fix](nestedType)fix nested data type to create table (#26506) 2023-11-09 14:38:41 +08:00
baae7bf339 [fix](information_schema)fix bug that metadata_name_ids error tableid and append information_schema case. (#26238)
fix bug that  #24059 .
Added some information_schema scanner tests.
files
schema_privileges
table_privileges
partitions
rowsets
statistics
table_constraints

Based on infodb_support_ext_catalog=false, it currently includes tests for all tables under the information_schema database.
2023-11-09 14:07:12 +08:00
22bf2889e5 [feature](tvf)(jni-avro)jni-avro scanner add complex data types (#26236)
Support avro's enum, record, union data types
2023-11-09 13:58:49 +08:00
5f62a4462d [Enhancement](wal) Add wal space back pressure (#26483) 2023-11-09 12:29:05 +08:00
84d1c3ba30 [Improve](ddl) Use RecycleBin to force drop db/table asynchronously (#26563) 2023-11-09 12:28:16 +08:00
0929ad568f [fix](regression) move fault-injection data to the right place (#26618)
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
2023-11-09 12:22:13 +08:00
db317841a0 [hotfix](editlog) Fix upsert replay on follower not contains loadedTableIndexIds (#26597)
Signed-off-by: Jack Drogon <jack.xsuperman@gmail.com>
2023-11-09 12:21:43 +08:00
124a8a9b34 [enhancement](regression) add profile before datev2 insert for debug (#26617)
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
2023-11-09 12:21:15 +08:00
33e46ee13d [enhancement](config) enable single_replica_load by default in BE (#26619)
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
2023-11-09 12:14:37 +08:00
5d52162484 [Test](statistics) Add test cases for external table statistics (#26511)
1. Test for close and open auto collection for external catalog.
2. Test for analyze table table_name (column) and whole table.
2023-11-09 12:12:29 +08:00
57ed781bb6 [fix](regression-test) Add tvf regression tests (#26455) 2023-11-09 12:09:32 +08:00
d1438a8563 [Fix](orc-reader) Fix orc complex types when late materialization was turned on by disabling late materialization in this case. (#26548)
Fix orc complex types when late materialization was turned on in orc reader by disabling late materialization in this case.
2023-11-09 12:05:43 +08:00