Jerry Hu 848234bd92 [fix](exchange) query blocked by exchange reach limit (#42853) ...

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->

2024-10-30 10:13:26 +08:00

..

aggregation_sink_operator.cpp

[cherry-pick](branch-2.1) Improve local shuffle strategy (#40030)

2024-08-29 14:16:16 +08:00

aggregation_sink_operator.h

[pick](branch-2.1) pick #41555 #41592 #38204 (#41781)

2024-10-14 14:05:08 +08:00

aggregation_source_operator.cpp

[Bug](expr) execute expr should use local states instead of operators (#40189) (#40219)

2024-09-01 00:41:10 +08:00

aggregation_source_operator.h

[fix] fix wrong result of spill agg with limit (#35403)

2024-05-28 13:12:03 +08:00

analytic_sink_operator.cpp

[cherry-pick](branch-2.1) Improve local shuffle strategy (#40030)

2024-08-29 14:16:16 +08:00

analytic_sink_operator.h

[fix](analytic) Fix data distribution after analytic operator (#41902) (#41949)

2024-10-16 18:41:56 +08:00

analytic_source_operator.cpp

[fix](window_func) fix bug of agg function used in window function and add many test cases (#40678) (#41328)

2024-09-26 22:50:34 +08:00

analytic_source_operator.h

[refactor](pipelineX) Reduce prepare overhead (PART I) (#33550)

2024-04-17 23:42:12 +08:00

assert_num_rows_operator.cpp

[Bug](expr) execute expr should use local states instead of operators (#40189) (#40219)

2024-09-01 00:41:10 +08:00

assert_num_rows_operator.h

[Bug](expr) execute expr should use local states instead of operators (#40189) (#40219)

2024-09-01 00:41:10 +08:00

const_value_operator.h

[refactor](operator) We dont need OperatorBuilder to instantiate Operator (#29133)

2023-12-29 13:54:54 +08:00

data_queue.cpp

[chore](session_variable) Add 'data_queue_max_blocks' to prevent the DataQueue from occupying too much memory. (#34017) (#34395)

2024-05-05 21:20:33 +08:00

data_queue.h

[chore](session_variable) Add 'data_queue_max_blocks' to prevent the DataQueue from occupying too much memory. (#34017) (#34395)

2024-05-05 21:20:33 +08:00

datagen_operator.cpp

[cherry-pick](branch-2.1) Improve local shuffle strategy (#40030)

2024-08-29 14:16:16 +08:00

datagen_operator.h

[pipelineX](refactor) remove source state from operator functions (#31435)

2024-02-28 13:05:57 +08:00

distinct_streaming_aggregation_operator.cpp

[Bug](expr) execute expr should use local states instead of operators (#40189) (#40219)

2024-09-01 00:41:10 +08:00

distinct_streaming_aggregation_operator.h

[pick](branch-2.1) pick #41555 #41592 #38204 (#41781)

2024-10-14 14:05:08 +08:00

distinct_streaming_aggregation_sink_operator.cpp

[improve](distinct agg) add check of hash table to decide whether emplace value (#32063)

2024-03-15 18:06:15 +08:00

distinct_streaming_aggregation_sink_operator.h

[pipelineX](refactor) refactor streaming agg structure (#31151)

2024-02-21 13:53:32 +08:00

distinct_streaming_aggregation_source_operator.cpp

[improve](distinct agg) add check of hash table to decide whether emplace value (#32063)

2024-03-15 18:06:15 +08:00

distinct_streaming_aggregation_source_operator.h

[improve](distinct agg) add check of hash table to decide whether emplace value (#32063)

2024-03-15 18:06:15 +08:00

empty_set_operator.cpp

[pipelineX](refactor) remove source state from operator functions (#31435)

2024-02-28 13:05:57 +08:00

empty_set_operator.h

[pipelineX](refactor) remove source state from operator functions (#31435)

2024-02-28 13:05:57 +08:00

empty_source_operator.cpp

[Bug](memleak) Fix emptyoperator may cause node not close (#20525)

2023-06-07 01:27:13 +08:00

empty_source_operator.h

[Fix](Status) Handle returned overall Status correctly (#31692)

2024-03-09 19:44:39 +08:00

es_scan_operator.cpp

[fix](ES Catalog)Do not push down limit to ES when predicates can not be processed by ES. (#40111) (#40265)

2024-09-03 11:17:24 +08:00

es_scan_operator.h

[pipelineX](local shuffle) Support parallel execution despite of tablet number (#28266)

2023-12-14 12:53:54 +08:00

exchange_sink_buffer.cpp

[fix](exchange) query blocked by exchange reach limit (#42853)

2024-10-30 10:13:26 +08:00

exchange_sink_buffer.h

[debug](rpc) debug rpc time consumption problem (#39852)

2024-08-24 19:59:39 +08:00

exchange_sink_operator.cpp

[pick](branch-2.1) pick #41292 #41350 #41589 #41628 #41743 #41601 #41667 #41751 (#41927)

2024-10-16 15:41:28 +08:00

exchange_sink_operator.h

[pick](branch-2.1) pick #41292 #41350 #41589 #41628 #41743 #41601 #41667 #41751 (#41927)

2024-10-16 15:41:28 +08:00

exchange_source_operator.cpp

[cherry-pick](branch-21)fix the exchange not skip offset rows (#42190) (#42330)

2024-10-24 11:08:17 +08:00

exchange_source_operator.h

[pipelineX](refactor) Rebuild relationship between dep and operator (#31487)

2024-02-29 12:38:03 +08:00

file_scan_operator.cpp

[opt](catalog) merge scan range to avoid too many splits (#38311) (#38964)

2024-08-06 21:57:02 +08:00

file_scan_operator.h

[opt](split) get file splits in batch mode (#34032) (#35107)

2024-05-21 22:27:07 +08:00

group_commit_block_sink_operator.h

[refactor](operator) We dont need OperatorBuilder to instantiate Operator (#29133)

2023-12-29 13:54:54 +08:00

hashjoin_build_sink.cpp

[pick](branch-2.1) pick #41292 #41350 #41589 #41628 #41743 #41601 #41667 #41751 (#41927)

2024-10-16 15:41:28 +08:00

hashjoin_build_sink.h

[pick](branch-2.1) pick #41555 #41592 #38204 (#41781)

2024-10-14 14:05:08 +08:00

hashjoin_probe_operator.cpp

[opt](join) check datatype of intermediate slots in hash join (#38556) (#38792)

2024-08-05 09:03:21 +08:00

hashjoin_probe_operator.h

[pick](branch-2.1) pick #41555 #41592 #38204 (#41781)

2024-10-14 14:05:08 +08:00

hive_table_sink_operator.cpp

[Feature-WIP](multi-catalog) Add Hive sink on BE side. (#32306) (#32364)

2024-03-18 11:23:01 +08:00

hive_table_sink_operator.h

[Feature-WIP](multi-catalog) Add Hive sink on BE side. (#32306) (#32364)

2024-03-18 11:23:01 +08:00

iceberg_table_sink_operator.cpp

[Feature](iceberg-writer) Implements iceberg sink basic functionality for inserting into table. (#35587)

2024-05-29 16:40:54 +08:00

iceberg_table_sink_operator.h

[Feature](iceberg-writer) Implements iceberg sink basic functionality for inserting into table. (#35587)

2024-05-29 16:40:54 +08:00

jdbc_scan_operator.cpp

[cherry-pick](scan)scanner could eos early when reached limit (#36535) (#36736)

2024-06-25 17:22:43 +08:00

jdbc_scan_operator.h

[pipelineX](local shuffle) Support parallel execution despite of tablet number (#28266)

2023-12-14 12:53:54 +08:00

jdbc_table_sink_operator.cpp

[pipelineX](refactor) remove source state from operator functions (#31435)

2024-02-28 13:05:57 +08:00

jdbc_table_sink_operator.h

[pipelineX](refactor) remove source state from operator functions (#31435)

2024-02-28 13:05:57 +08:00

join_build_sink_operator.cpp

[Improvement](runtime-filter) support sync join node build side's size to init bloom runtime filter (#32180)

2024-04-11 09:31:50 +08:00

join_build_sink_operator.h

[Improvement](runtime-filter) support sync join node build side's size to init bloom runtime filter (#32180)

2024-04-11 09:31:50 +08:00

join_probe_operator.cpp

[refactor](pipelineX) Reduce prepare overhead (PART III) (#33689)

2024-04-17 23:42:14 +08:00

join_probe_operator.h

[refactor](pipelineX) Reduce prepare overhead (PART III) (#33689)

2024-04-17 23:42:14 +08:00

meta_scan_operator.cpp

[cherry-pick](scan)scanner could eos early when reached limit (#36535) (#36736)

2024-06-25 17:22:43 +08:00

meta_scan_operator.h

[improvement](scan) avoid too many scanners for file scan node (#25727)

2023-10-29 17:41:31 +08:00

multi_cast_data_stream_sink.cpp

[feature](spill) Implement spill to disk for hash join, aggregation and sort for pipelineX (#31910)

2024-03-12 14:12:09 +08:00

multi_cast_data_stream_sink.h

[pick](branch-2.1) pick #41292 #41350 #41589 #41628 #41743 #41601 #41667 #41751 (#41927)

2024-10-16 15:41:28 +08:00

multi_cast_data_stream_source.cpp

[opt](MultiCast) Avoid copying while holding a lock (#37462) (#39816)

2024-08-23 17:15:34 +08:00

multi_cast_data_stream_source.h

[Bug](expr) execute expr should use local states instead of operators (#40189) (#40219)

2024-09-01 00:41:10 +08:00

multi_cast_data_streamer.cpp

[opt](MultiCast) Avoid copying while holding a lock (#37462) (#39816)

2024-08-23 17:15:34 +08:00

multi_cast_data_streamer.h

[opt](MultiCast) Avoid copying while holding a lock (#37462) (#39816)

2024-08-23 17:15:34 +08:00

mysql_scan_operator.cpp

[Fix](Status) Handle returned overall Status correctly (#31692)

2024-03-09 19:44:39 +08:00

mysql_scan_operator.h

[refactor](operator) We dont need OperatorBuilder to instantiate Operator (#29133)

2023-12-29 13:54:54 +08:00

nested_loop_join_build_operator.cpp

[refactor](pipelineX) Reduce prepare overhead (PART III) (#33689)

2024-04-17 23:42:14 +08:00

nested_loop_join_build_operator.h

[refactor](pipelineX) Reduce prepare overhead (PART III) (#33689)

2024-04-17 23:42:14 +08:00

nested_loop_join_probe_operator.cpp

[bugifx](core) fix logical error of status check in nestedloop join (#35365)

2024-05-25 17:46:44 +08:00

nested_loop_join_probe_operator.h

[refactor](pipelineX) Reduce prepare overhead (PART III) (#33689)

2024-04-17 23:42:14 +08:00

olap_scan_operator.cpp

[opt](scanner profile) Rename some filed name to keep consistent with audit log. #41993 (#42137)

2024-10-25 01:05:13 +08:00

olap_scan_operator.h

[Chore](inverted index) remove useless code of compound filters for inverted index #40258 (#41448)

2024-09-29 17:27:29 +08:00

olap_table_sink_operator.cpp

[refactor](pipelineX) Reduce prepare overhead (PART II) (#33681)

2024-04-17 23:42:13 +08:00

olap_table_sink_operator.h

[refactor](pipelineX) Reduce prepare overhead (PART II) (#33681)

2024-04-17 23:42:13 +08:00

olap_table_sink_v2_operator.cpp

[refactor](pipelineX) Reduce prepare overhead (PART II) (#33681)

2024-04-17 23:42:13 +08:00

olap_table_sink_v2_operator.h

[refactor](pipelineX) Reduce prepare overhead (PART II) (#33681)

2024-04-17 23:42:13 +08:00

operator.cpp

[pipelineX](refactor) refactor streaming agg structure (#31151)

2024-02-21 13:53:32 +08:00

operator.h

[pick](branch-2.1) pick #41555 #41592 #38204 (#41781)

2024-10-14 14:05:08 +08:00

partition_sort_sink_operator.cpp

[cherry-pick](branch-21) pick (#39057) (#41352) (#41958)

2024-10-17 14:30:40 +08:00

partition_sort_sink_operator.h

[cherry-pick](branch-21) pick (#39057) (#41352) (#41958)

2024-10-17 14:30:40 +08:00

partition_sort_source_operator.cpp

[cherry-pick](branch-21) pick (#39057) (#41352) (#41958)

2024-10-17 14:30:40 +08:00

partition_sort_source_operator.h

[cherry-pick](branch-21) pick (#39057) (#41352) (#41958)

2024-10-17 14:30:40 +08:00

partitioned_aggregation_sink_operator.cpp

[cherry-pick](branch-2.1) Improve local shuffle strategy (#40030)

2024-08-29 14:16:16 +08:00

partitioned_aggregation_sink_operator.h

[pick](branch-2.1) pick #41292 #41350 #41589 #41628 #41743 #41601 #41667 #41751 (#41927)

2024-10-16 15:41:28 +08:00

partitioned_aggregation_source_operator.cpp

[branch-2.1](memory) Refactor refresh workload groups weighted memory ratio and record refresh interval memory growth (#39760)

2024-08-22 17:33:11 +08:00

partitioned_aggregation_source_operator.h

[improvement](spill) improve cancel (#34451)

2024-05-07 00:07:20 +08:00

partitioned_hash_join_probe_operator.cpp

[branch-2.1](memory) Refactor refresh workload groups weighted memory ratio and record refresh interval memory growth (#39760)

2024-08-22 17:33:11 +08:00

partitioned_hash_join_probe_operator.h

[pick](branch-2.1) pick #41555 #41592 #38204 (#41781)

2024-10-14 14:05:08 +08:00

partitioned_hash_join_sink_operator.cpp

[branch-2.1](memory) Refactor refresh workload groups weighted memory ratio and record refresh interval memory growth (#39760)

2024-08-22 17:33:11 +08:00

partitioned_hash_join_sink_operator.h

[pick](branch-2.1) pick #41555 #41592 #38204 (#41781)

2024-10-14 14:05:08 +08:00

repeat_operator.cpp

[Bug](expr) execute expr should use local states instead of operators (#40189) (#40219)

2024-09-01 00:41:10 +08:00

repeat_operator.h

[refactor](pipelineX) Reduce prepare overhead (PART I) (#33550)

2024-04-17 23:42:12 +08:00

result_file_sink_operator.cpp

[pick](branch-2.1) pick #41676 #41740 #41857 (#41904)

2024-10-15 22:41:17 +08:00

result_file_sink_operator.h

[pipelineX](refactor) remove source state from operator functions (#31435)

2024-02-28 13:05:57 +08:00

result_sink_operator.cpp

[chore](log) Log written rows when result sink close #39647 (#40011)

2024-08-29 16:49:44 +08:00

result_sink_operator.h

[branch-2.1](arrow-flight-sql) Add config arrow_flight_result_sink_buffer_size_rows (#38223)

2024-07-24 15:15:39 +08:00

scan_operator.cpp

[opt](scanner profile) Rename some filed name to keep consistent with audit log. #41993 (#42137)

2024-10-25 01:05:13 +08:00

scan_operator.h

[opt](scanner profile) Rename some filed name to keep consistent with audit log. #41993 (#42137)

2024-10-25 01:05:13 +08:00

schema_scan_operator.cpp

[pick](branch-2.1) pick #41555 #41592 #38204 (#41781)

2024-10-14 14:05:08 +08:00

schema_scan_operator.h

[Improvement](schema scan) Use async scanner for schema scanners (#38… (#38666)

2024-08-01 16:05:24 +08:00

select_operator.cpp

…

select_operator.h

[pipelineX](refactor) remove source state from operator functions (#31435)

2024-02-28 13:05:57 +08:00

set_probe_sink_operator.cpp

[Fix](compile) Fix gcc compile on master (#33864)

2024-04-19 23:41:37 +08:00

set_probe_sink_operator.h

[fix](local shuffle) Fix hash shuffle local exchanger (#40101)

2024-08-29 16:52:28 +08:00

set_sink_operator.cpp

[Bug](fix) fix coredump case in (not null, null) execpt (not null, not null) case (#38756)

2024-08-04 10:44:10 +08:00

set_sink_operator.h

[fix](local shuffle) Fix hash shuffle local exchanger (#40101)

2024-08-29 16:52:28 +08:00

set_source_operator.cpp

[fix](set) incorrect result of set operator (#35607)

2024-05-30 19:59:37 +08:00

set_source_operator.h

[pipelineX](refactor) Rebuild relationship between dep and operator (#31487)

2024-02-29 12:38:03 +08:00

sort_sink_operator.cpp

[cherry-pick](branch-2.1) Improve local shuffle strategy (#40030)

2024-08-29 14:16:16 +08:00

sort_sink_operator.h

[pick](branch-2.1) pick #41555 #41592 #38204 (#41781)

2024-10-14 14:05:08 +08:00

sort_source_operator.cpp

[feature](spill) Implement spill to disk for hash join, aggregation and sort for pipelineX (#31910)

2024-03-12 14:12:09 +08:00

sort_source_operator.h

[feature](spill) Implement spill to disk for hash join, aggregation and sort for pipelineX (#31910)

2024-03-12 14:12:09 +08:00

spill_sort_sink_operator.cpp

[cherry-pick](branch-2.1) Improve local shuffle strategy (#40030)

2024-08-29 14:16:16 +08:00

spill_sort_sink_operator.h

[pick](branch-2.1) pick #41292 #41350 #41589 #41628 #41743 #41601 #41667 #41751 (#41927)

2024-10-16 15:41:28 +08:00

spill_sort_source_operator.cpp

[branch-2.1](memory) Refactor refresh workload groups weighted memory ratio and record refresh interval memory growth (#39760)

2024-08-22 17:33:11 +08:00

spill_sort_source_operator.h

[improvement](spill) improve cancel (#34451)

2024-05-07 00:07:20 +08:00

streaming_aggregation_operator.cpp

[Bug](expr) execute expr should use local states instead of operators (#40189) (#40219)

2024-09-01 00:41:10 +08:00

streaming_aggregation_operator.h

[fix](stream agg) fix coredump when close if open failed (#33978)

2024-04-24 17:13:50 +08:00

streaming_aggregation_sink_operator.cpp

[pipelineX](refactor) refactor streaming agg structure (#31151)

2024-02-21 13:53:32 +08:00

streaming_aggregation_sink_operator.h

[pipelineX](refactor) refactor streaming agg structure (#31151)

2024-02-21 13:53:32 +08:00

streaming_aggregation_source_operator.cpp

[feature](spill) Implement spill to disk for hash join, aggregation and sort for pipelineX (#31910)

2024-03-12 14:12:09 +08:00

streaming_aggregation_source_operator.h

[pipelineX](refactor) refactor streaming agg structure (#31151)

2024-02-21 13:53:32 +08:00

table_function_operator.cpp

[opt](Nereids) use 1 instead narrowest column when do column pruning (#41548) (#41627)

2024-10-10 14:02:23 +08:00

table_function_operator.h

[refactor](pipelineX) Reduce prepare overhead (PART I) (#33550)

2024-04-17 23:42:12 +08:00

table_sink_operator.h

[refactor](operator) We dont need OperatorBuilder to instantiate Operator (#29133)

2023-12-29 13:54:54 +08:00

union_sink_operator.cpp

[chore](session_variable) Add 'data_queue_max_blocks' to prevent the DataQueue from occupying too much memory. (#34017) (#34395)

2024-05-05 21:20:33 +08:00

union_sink_operator.h

[pick](branch-2.1) pick #41555 #41592 #38204 (#41781)

2024-10-14 14:05:08 +08:00

union_source_operator.cpp

[pipeline](fix) Avoid to use a freed dependency when cancelled (#34584) (#38046)

2024-07-18 15:27:10 +08:00

union_source_operator.h

[pick](branch-2.1) pick #41555 #41592 #38204 (#41781)

2024-10-14 14:05:08 +08:00