database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
17,549 Commits 1 Branch 0 Tags
ea2fbfaffaf9cd09c3cfa8eabbde415dfdbeeade
Commit Graph

7205 Commits

Author SHA1 Message Date
yiguolei
62023d705d [refactor](rename) rename task group to workload group in be (#32204) 
---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
 2024-03-15 18:04:02 +08:00
lw112
0578b28d54 [fix](function) fixed the get_json_string function (#32150) 2024-03-15 18:04:02 +08:00
Mryange
4534300030 [fix](Operator) RepeatNode does not handle empty expressions. (#32112) 
In the past, RepeatNode did not handle empty expressions.
It used DCHECK to check if the expression was non-empty.
In non-debug mode, this caused _child_block to remain unprocessed, resulting in a deadlock.
Now, if the expression is empty, the output block directly outputs _child_block
 2024-03-15 18:02:33 +08:00
HHoflittlefish777
56a14c912a [fix](routineload) fix consume data too slow in partial partitions (#32126) 2024-03-15 18:01:22 +08:00
lihangyu
554f13ca12 [Fix](Variant) fix inverted index lost due to wrong column unique id (#31992) 2024-03-15 18:01:22 +08:00
lihangyu
1db57c0667 [Optimization][Scanner] Skip _init_variant_columns when there are no variant columns, and ensure inherit_tablet_index is called only once (#32174) 2024-03-15 18:01:19 +08:00
xy
e6f9bd4023 [Opt](exec) Reduce unnecessary find function calls (#32165) 2024-03-15 18:00:57 +08:00
Xin Liao
dc687dc4cc [fix](merge-clod) fix file not found when load for mow table (#32144) 2024-03-15 18:00:24 +08:00
zhangstar333
94a75c27e7 [feature](pipelineX) support paritition tablet sink shuffle (#31689) 2024-03-15 17:58:01 +08:00
Xinyi Zou
7b74b199a5 [fix](memory) Fix LRU cache deleter and memory tracking (#32080) 
In order to add common code to the value deleter of LRU cache, let all lru cache values inherit from LRUCacheValueBase class and tracking memory in destructor.
 2024-03-15 17:57:58 +08:00
wangbo
df5ec16d7c [Refactor](exectuor)Add schema type table active_queries (#32057) 
* Add schema type table active_queries
 2024-03-15 17:57:28 +08:00
plat1ko
6f0aa5fb84 [bug](s3) Fix object data is overwritten by empty object (#32258) 2024-03-15 10:18:45 +08:00
zy-kkk
bbdce3eb5e [fix](jdbc catalog) fix jdbc-connector coredump as get env return nullptr (#32217) 2024-03-14 16:05:08 +08:00
yiguolei
20d6698c27 [bugfix](arm compile) could not compile on arm because -Werror=maybe-uninitialized 2024-03-14 12:11:25 +08:00
zclllyybb
847ec368be [Fix](smooth-upgrade) Fix incompatibility when upgrade from 2.0 to 2.1 (#32220) 2024-03-14 11:23:05 +08:00
HappenLee
b031c95324 [Opt](exec) use libbase64 to replace base64 code in doris (#32078) 
* [Opt](exec) use libbase64 to replace base64 code in doris
 2024-03-14 09:20:50 +08:00
HappenLee
f2404ff553 [Opt](exec) remove the unless mem alloc in base64 (#32019) 2024-03-14 09:19:40 +08:00
Pxl
6b08a4ec93 [Bug](top-n) do not get runtime predicate when predicate not initialized #32209 2024-03-14 09:12:09 +08:00
Pxl
2f4401189a [Bug](top-n) do not update topn filter when sort node and scan node are not in the… (#32159) 2024-03-13 16:21:36 +08:00
Pxl
fa111c371b [Chore](top-n) check runtime predicate inited when scan operator open (#32140) 
check runtime predicate inited when scan operator open
 2024-03-12 22:52:10 +08:00
yiguolei
5da7cd0fba [bugfix](becore) has to use value to capture lambda value to avoid core during callback (#32132) 
Co-authored-by: yiguolei <yiguolei@gmail.com>
 2024-03-12 22:51:44 +08:00
yiguolei
0159a75ced [bugfix](becore) be will core when stop because the map is modified during iterator (#32105) 
Co-authored-by: yiguolei <yiguolei@gmail.com>
 2024-03-12 18:50:26 +08:00
Jerry Hu
473bd3ee64 [fix](function) incorrect result of eq_for_null (#32103) 2024-03-12 18:50:26 +08:00
HHoflittlefish777
6610307eb0 [opt](routine-load) end Kafka consume when meets partition EOF #32046 2024-03-12 18:50:06 +08:00
qiye
5f125bbaaa [improvement](binlog)Support inverted index in CCR (#31743) (#32101) 2024-03-12 15:34:08 +08:00
Gabriel
61928f7df5 [pipelineX](scanner) Use the actual instances num when ignore data distribution (#32081) 2024-03-12 14:20:39 +08:00
AlexYue
ffa904c487 [enhance](Cooldown) Skip cooldown if the tablet is dropped (#32079) 2024-03-12 14:20:39 +08:00
TengJianPing
b41b17ad0a [fix](spill) fix storage engine type cast error (#32071) 2024-03-12 14:20:18 +08:00
Tiewei Fang
aea9ddc3cb [Fix](Outfiel) fix be core when the open method of vfile_result_writer failed #32042 2024-03-12 14:20:18 +08:00
Xinyi Zou
4268634115 [fix](memory) Fix Allocator cancel pipelinex query #32048 2024-03-12 14:20:18 +08:00
Benjaminwei
ccd21a6ea4 [Improve](InPredict) enhance in predict with array type (#31828) 2024-03-12 14:19:14 +08:00
Mryange
68a5319da3 [fix](pipelineX) _local_channel_dependency is null in non pipelineX (#32054) 2024-03-12 14:19:04 +08:00
zy-kkk
fe3988c686 [fix](compile) fix BE compile failure on Mac (#32044) 2024-03-12 14:17:55 +08:00
Mryange
c0f2d0188b [feature](pipelineX) add mem control in local exchange sink (#31982) 2024-03-12 14:17:48 +08:00
Mryange
b0b7161ad0 [feature](rf) add filter info profile when rf run as expr (#31822) 2024-03-12 14:17:48 +08:00
HappenLee
2470634859 [RuntimeFilter] fix <=> runtime filter failed bug (#32003) 2024-03-12 14:13:13 +08:00
TengJianPing
3358f76a7f [feature](spill) Implement spill to disk for hash join, aggregation and sort for pipelineX (#31910) 
Co-authored-by: Jerry Hu <mrhhsg@gmail.com>
 2024-03-12 14:12:09 +08:00
Gabriel
ae6003504c [pipelineX](partition sort) Add some nessacery metrics (#32020) 
Add some necessary metrics
 2024-03-09 19:55:48 +08:00
wangbo
c5390d00bb [Improvement]Add schema table backend_active_tasks (#31945) 2024-03-09 19:55:48 +08:00
Pxl
38caa35a73 [Chore](third-party) update clucene version (#32014) 
update clucene version
 2024-03-09 19:55:48 +08:00
TengJianPing
d2e7a68d11 [enhancement](util) print if using nereids planner when be coredump (#31981) 2024-03-09 19:55:47 +08:00
amory
621803c547 [FIX](InPredict) fix in params in to context for thread fragment query (#31935) 2024-03-09 19:45:46 +08:00
HappenLee
6ef4ab631d [Opt](func) reduce the useless mem alloc and const opt the concat code (#31983) 2024-03-09 19:45:46 +08:00
abmdocrt
609761567c [Fix](partial-update) Fix wrong column number passing to BE when partial and enable nereids (#31461) 
* Problem:
Inconsistent behavior occurs when executing partial column update `UPDATE` statements and `INSERT` statements on merge-on-write tables with the Nereids optimizer enabled. The number of columns passed to BE differs; `UPDATE` operations incorrectly pass all columns, while `INSERT` operations correctly pass only the updated columns.

Reason:
The Nereids optimizer does not handle partial column update `UPDATE` statements properly. The processing logic for `UPDATE` statements rewrites them as equivalent `INSERT` statements, which are then processed according to the logic of `INSERT` statements. For example, assuming a MoW table structure with columns k1, k2, v1, v2, the correct rewrite should be:
* `UPDATE` table t1 set v1 = v1 + 1 where k1 = 1 and k2 = 2
 * =>
 * `INSERT` into table (v1) select v1 + 1 from table t1 where k1 = 1 and k2 = 2

However, the actual rewriting process does not consider the logic for partial column updates, leading to all columns being included in the `INSERT` statement, i.e., the result is:
* `INSERT` into table (k1, k2, v1, v2) select k1, k2, v1 + 1, v2 from table t1 where k1 = 1 and k2 = 2

This results in `UPDATE` operations incorrectly passing all columns to BE.

Solution:
Having analyzed the cause, the solution is straightforward: when rewriting partial column update `UPDATE` statements to `INSERT` statements, only retain the updated columns and all key columns (as partial column updates must include all key columns). Additionally, this PR includes error injection cases to verify the number of columns passed to BE is correct.

* 2

* 3

* 4

* 5
 2024-03-09 19:45:42 +08:00
lihangyu
e8aa5ee7d5 [Improve](Variant) support bloom filter for variant subcolumns (#31347) 
* [Improve](Variant) support bloom filter for variant subcolumns

* rebase
 2024-03-09 19:45:03 +08:00
Gabriel
f968d96545 [profile](pipelineX) Add lost metrics (#31964) 2024-03-09 19:45:03 +08:00
Jerry Hu
93d298d34a [fix](agg) wrong result of two or more map_agg functions in query (#31928) 2024-03-09 19:45:03 +08:00
Gabriel
b2de83f250 [agg](conf) Add a knob to control distinct agg (#31930) 
Add a knob to control distinct agg
 2024-03-09 19:44:54 +08:00
lihangyu
0da010603e [Improve](TabletSchemaCache) reduce duplicated memory consumption for column name and column path (#31141) 
Both could be reference to related field in TabletColumn.And use shared_ptr for TabletColumn in TabletSchema for later memory reuse
 2024-03-09 19:44:42 +08:00
Uniqueyou
779ca464a5 [Fix](Status) Handle returned overall Status correctly (#31692) 
Handle returned overall Status correctly
 2024-03-09 19:44:39 +08:00