abbd2cedff
[fix](Nereids) merge limit should use bottom phase ( #29142 )
2023-12-27 21:04:00 +08:00
0436013baf
[fix](decimal) fix cast decimal overflow and add test cases for casting decimalv2 to decimalv3 ( #29165 )
2023-12-27 20:58:37 +08:00
5f71691401
[fix](read) fix unexpected overflow of uninitialized column data in VStatisticsIterator::next_batch ( #29141 )
2023-12-27 20:58:02 +08:00
9715db61d4
[FIX](complextype)fix count func with complex type ( #28873 )
2023-12-27 20:38:44 +08:00
6d817bc253
[fix](topn opt) avoid using topn runtime predicate which segment does not contain such column(column unique id) when pruning segment ( #29148 )
2023-12-27 20:31:03 +08:00
286af3bce9
[doc](file-cache) fix wrong config ( #29156 )
...
* [doc](file-cache) fix wrong config
* 2
2023-12-27 18:51:01 +08:00
d70743a2fc
[fix](doc) spell errors fixes ha-action.md ( #28155 )
2023-12-27 16:57:07 +08:00
c1303e9eee
[fix](doc) example case fixed in best practices ( #28156 )
2023-12-27 16:56:44 +08:00
522e48c764
[fix](doc) spell errors fixes for be-olap-error-code docs ( #27866 )
2023-12-27 16:55:59 +08:00
ef8ae75e37
[fix](doc) updated node version as per given in the url. ( #28427 )
2023-12-27 16:55:37 +08:00
cec7eba1cc
[fix](doc) spell errors fixes for table-repair and table local debug ( #28153 )
2023-12-27 16:51:58 +08:00
4269041563
[enhancement](Nereids): catch exception when calling getConstraintMap for external map ( #29112 )
2023-12-27 16:45:03 +08:00
1c0bbcf416
[fix](doc) typo fix in sql-interception docs ( #28445 )
2023-12-27 16:42:43 +08:00
64ca508c0b
[fix](doc) typo fix in schema change docs ( #28453 )
2023-12-27 16:42:28 +08:00
26842049f3
[fix](doc) typo fix in fqdn page ( #28509 )
2023-12-27 16:42:00 +08:00
c75e63a2a5
[Improvement](scan) Use scanner to do projection of scan node ( #29124 )
2023-12-27 16:00:52 +08:00
cd1e109cc3
[debug string](pipeline) Add necessary debug info ( #29119 )
2023-12-27 15:57:22 +08:00
2d2f14bc75
[fix](paimon) use SlotDescriptor to parse the required fields ( #28990 )
...
Before this PR, Paimon has created the schema of `VectorTable` by accessing meta information. However, once the schema of `VectorTable` in java is not same as `Block` in c++, BE will crashed, and there is no good way to troubleshoot errors.
2023-12-27 15:45:53 +08:00
cfed36afbf
[Fix](topn opt) prevent from merge __TEMP__ column in segment iterator ( #29121 )
2023-12-27 15:42:48 +08:00
0607760efa
[typo](resource) fix create odbc resource error msg ( #29019 )
2023-12-27 15:30:52 +08:00
8035287010
[fix](doc) spell errors fixes for memory tracker docs ( #27865 )
2023-12-27 15:22:43 +08:00
d3e1afa2e4
[fix](doc) typo fix in metaInfoActionV2 page ( #28510 )
2023-12-27 15:21:20 +08:00
e0014308bd
[fix](load) Recycle progress before removing bulk load job ( #29066 )
2023-12-27 15:17:04 +08:00
578a7dadc1
[improvement](statistics)Use key column id to query column statistic table. ( #28885 )
...
Use id as where predicate to load column statistic cache. This could improve performance, because id is the first order key in column statistics table.
2023-12-27 14:38:55 +08:00
fc6a587352
[test](Nereids) add subquery pull up rule group ut ( #26725 )
...
add subquery pull up rule group ut:
single table part
(1) subquery_basic_pullup_basic: basic part
(2) subquery_basic_pullup_uk: using unique key
(3) subquery_basic_pullup_and: with and
(4) subquery_basic_pullup_or: with or
(5) subquery_basic_pullup_and_subquery: with and subquery
(6) subquery_basic_pullup_or_subquery: with or subquery
multi table part
(1) subquery_multitable_pullup_basic: basic part
(2) subquery_multitable_pullup_and: with and
(3) subquery_multitable_pullup_or: with or
(4) subquery_multitable_pullup_and_subquery: with and subquery
(5) subquery_multitable_pullup_or_subquery: with or subquery
top op related, such as group by/having, order by, win func, select list:
(1) subquery_topop_pullup_groupby: group by
(2) subquery_topop_pullup_having: having
(3) subquery_topop_pullup_orderby: order by
(4) subquery_topop_pullup_winfunc: window function
(5) subquery_topop_pullup_selectlist: select list
misc, such as group by correlated, view, dml, etc:
(1) subquery_misc_pullup_misc: group by correlated, view, etc
(2) subquery_misc_pullup_dml: insert/update/delete
2023-12-27 14:03:53 +08:00
0ac9b3d113
[Fix](Job)cancel task is not cleared from running task ( #29114 )
2023-12-27 13:59:12 +08:00
05f185ff44
[fix][compile]unused variable ( #28992 )
2023-12-27 13:00:43 +08:00
51bd402b58
[fix](planner) Fix delete statement on MOR unique table ( #28968 )
...
add conditions for mor table removed in #26776
2023-12-27 12:59:15 +08:00
e33395a0ee
[fix](paimon)Remove the static attribute of the source for paimon ( #29032 )
2023-12-27 12:57:39 +08:00
6f5672f318
[Refact](inverted index) refactor inverted index writer init ( #29072 )
2023-12-27 12:49:26 +08:00
6ed09c3f46
[fix](mtmv)add log for resolve pending task ( #29078 )
2023-12-27 12:36:17 +08:00
0b5fe681e4
[fix](paimon) read batch by doris' batch size ( #29039 )
2023-12-27 12:35:17 +08:00
3e5c8d9949
[fix](read) remove logic of estimating count of rows to read in segment iterator to avoid wrong result of unique key. ( #29109 )
2023-12-27 12:25:14 +08:00
17f92155d7
[opt](Nerieds) add infer props to expression ( #28953 )
2023-12-27 12:21:25 +08:00
86a0eb9344
[fix](doc) spell errors fixes query profile docs ( #28437 )
2023-12-27 12:19:01 +08:00
9ff8bd2e9c
[Enhancement](Wal)Support dynamic wal space limit ( #27726 )
2023-12-27 11:51:32 +08:00
8e358617c7
[opt](nereids)expr normalize after filter pushdown ( #28743 )
...
normalize expression after filter push down
2023-12-27 11:37:36 +08:00
576a2b33cb
[fix](nereids) "not is null" stats estimation fix ( #28860 )
...
* fix not is null stats
2023-12-27 11:36:12 +08:00
58e7ad82b5
[feature](Nereids): support infer join when comapring mv ( #28988 )
2023-12-27 10:40:44 +08:00
6d26aca4ca
[fix](pipeline) sort_merge should throw exception in has_next_block if got failed status ( #29076 )
...
Test in regression-test/suites/datatype_p0/decimalv3/test_decimalv3_overflow.groovy::249 sometimes failed when there are multiple BEs and FE process report status slowly for some reason.
explain select k1, k2, k1 * k2 from test_decimal128_overflow2 order by 1,2,3
--------------
+----------------------------------------------------------------------------------------------------------------------------+
| Explain String(Nereids Planner) |
+----------------------------------------------------------------------------------------------------------------------------+
| PLAN FRAGMENT 0 |
| OUTPUT EXPRS: |
| k1[#5 ] |
| k2[#6 ] |
| (k1 * k2)[#7 ] |
| PARTITION: UNPARTITIONED |
| |
| HAS_COLO_PLAN_NODE: false |
| |
| VRESULT SINK |
| MYSQL_PROTOCAL |
| |
| 111:VMERGING-EXCHANGE |
| offset: 0 |
| |
| PLAN FRAGMENT 1 |
| |
| PARTITION: HASH_PARTITIONED: k1[#0 ], k2[#1 ] |
| |
| HAS_COLO_PLAN_NODE: false |
| |
| STREAM DATA SINK |
| EXCHANGE ID: 111 |
| UNPARTITIONED |
| |
| 108:VSORT |
| | order by: k1[#5 ] ASC, k2[#6 ] ASC, (k1 * k2)[#7 ] ASC |
| | offset: 0 |
| | |
| 102:VOlapScanNode |
| TABLE: regression_test_datatype_p0_decimalv3.test_decimal128_overflow2(test_decimal128_overflow2), PREAGGREGATION: ON |
| partitions=1/1 (test_decimal128_overflow2), tablets=8/8, tabletList=22841,22843,22845 ... |
| cardinality=6, avgRowSize=0.0, numNodes=1 |
| pushAggOp=NONE |
| projections: k1[#0 ], k2[#1 ], (k1[#0 ] * k2[#1 ]) |
| project output tuple id: 1 |
+----------------------------------------------------------------------------------------------------------------------------+
36 rows in set (0.03 sec)
Why failed:
Multiple BEs
Fragments 0 and 1 are MUST on different BEs
Pipeline task of VOlapScanNode which executes k1*k2 failed sets query status to cancelled
Pipeline task of VSort call try close, send Cancelled status to VMergeExchange
sort_curso did not throw exception when it meets error
2023-12-27 10:06:01 +08:00
a4e69f7272
[improvement](statistics)Remove retry load when load stats cache fail ( #28904 )
...
Remove retry load when load stats cache fail. This case usually happens when BE is down or BE OOM, retry doesn't work in these cases and may increase BE work load.
2023-12-27 09:59:34 +08:00
f6850f8cdb
Revert "[feature](load) enable memtable on sink node by default ( #28963 )" ( #29090 )
...
This reverts commit 17917a0adb57ac337dfd41e8853b1aa87c9d5acd.
2023-12-26 22:20:49 +08:00
a8e6676640
[Bug](security) BE download_files function exists log print sensitive msg #28592 ( #28594 )
2023-12-26 21:59:47 +08:00
ffa3c319dd
[fix](regression) Fix flaky test test_partial_update_2pc_schema_change ( #29037 )
2023-12-26 21:43:14 +08:00
c82155614b
[Bug] optimize the collection import Lists dependency packge #28523 ( #28579 )
2023-12-26 18:45:22 +08:00
ffc6596cef
[refactor](create tablet) default create tablet round robin ( #28911 )
2023-12-26 17:36:05 +08:00
6440fbfab6
[feature](scan) Implement parallel scanning by dividing the tablets based on the row range ( #28967 )
...
* [feature](scan) parallel scann on dup/mow mode
* fix bugs
2023-12-26 17:18:41 +08:00
4a60d01dc7
[improve](move-memtable) increase load_stream_flush_token_max_tasks ( #29011 )
2023-12-26 17:08:49 +08:00
1964a77d6c
[enhencement](config) change default memtable size & loadStreamPerNode & default load parallelism ( #28977 )
...
We change memtable size from 200MB to 100MB to achieve smoother flush
performance. We change loadStreamPerNode from 20 to 60 to avoid stream
rpc to be the bottleneck when enable memtable_on_sink_node. We change
default s3&broker load parallelsim to make the most of CPUs on moderm
multi-core systems.
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com >
2023-12-26 16:22:52 +08:00
d672e321cc
[feature](Nereids) support values inline table in query ( #28972 )
2023-12-26 15:51:08 +08:00
