f71fc3291f
[Bug](fix) right anti join error result when batch size is low ( #16510 )
2023-02-08 17:26:19 +08:00
f6a20f844b
[fix](hashjoin) join produce blocks with rows larger than batch size: handle join with other conjuncts ( #16402 )
2023-02-08 14:26:35 +08:00
91229bb87d
[Bug](makr join) Fix mark join with other conjuncts ( #16435 )
2023-02-07 09:31:41 +08:00
696c6ffcc5
[fix](join) crash caused by canceling query ( #16311 )
...
If the query was canceled,
the status in shared context may be `OK` with other fields not set.
2023-02-02 09:55:37 +08:00
bf16228851
[fix](hashjoin) join produce blocks with rows larger than batch size ( #16166 )
...
* [fix](hashjoin) join produce blocks with rows larger than batch size
* fix
2023-02-01 16:02:31 +08:00
46347a51d2
[Bug](exec) enable warning on ignoring function return value for vctx ( #16157 )
...
* enable warning on ignoring function return value for vctx
2023-01-29 17:23:21 +08:00
79ad74637d
[refactor](remove expr) remove non vectorized Expr and ExprContext related codes ( #16136 )
2023-01-24 10:45:35 +08:00
9f106161a7
[Bug](join) Fix null aware anti join error in fuzzy mode ( #15987 )
2023-01-17 11:32:16 +08:00
97fcad76f8
[enhancement](memtracker) Improve readability ( #15716 )
2023-01-16 16:30:35 +08:00
9468711f9f
[Bug](join) fix bug null aware left anti join not correct result ( #15841 )
2023-01-13 10:18:05 +08:00
d857b4af1b
[refactor](remove row batch) remove impala rowbatch structure ( #15767 )
...
* [refactor](remove row batch) remove impala rowbatch structure
Co-authored-by: yiguolei <yiguolei@gmail.com >
2023-01-11 09:37:35 +08:00
9c0f96883a
[fix](hashjoin) Fix right join pull output block memory overflow ( #15440 )
...
For outer join / right outer join / right semi join, when HashJoinNode::pull->process_data_in_hashtable outputs a block, it will output all rows of a key in the hash table into a block, and the output of a key is completed After that, it will check whether the block size exceeds the batch size, and if it exceeds, the output will be terminated.
If a key has 2000w+ rows, memory overflow will occur when the subsequent block operations on the 2000w+ rows are performed.
2023-01-10 10:10:43 +08:00
9c36278c4a
[improvement](pipeline) Support sharing hash table for broadcast join ( #15628 )
2023-01-06 15:11:28 +08:00
05d72e8919
[fix](join) fix anti join incorrectly outputs null values ( #15567 )
2023-01-06 09:55:48 +08:00
5ff5b8fc98
[feature](mark join) Support mark join for hash join node ( #15569 )
...
* [feature](mark join) Support mark join for hash join node
2023-01-05 09:32:26 +08:00
10be583e52
[chore](pipeline) optimize profile information ( #15433 )
2022-12-30 09:56:33 +08:00
06f71f2bca
[pipeline](fix) Fix bugs to pass all regression cases ( #15306 )
...
* [pipeline](fix) Fix bugs to pass all regression cases
* update
* update
2022-12-23 22:17:50 +08:00
b085ff49f0
[refactor](non-vec) delete non-vec data sink ( #15283 )
...
* [refactor](non-vec) delete non-vec data sink
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2022-12-23 14:10:47 +08:00
8ecf69b09b
[pipeline](regression) nested loop join test get error result in pipeline engine and refactor the code for need more input data ( #15208 )
2022-12-21 19:03:51 +08:00
af54299b26
[Pipeline](projection) Support projection on pipeline engine ( #15220 )
2022-12-21 15:47:29 +08:00
732417258c
[Bug](pipeline) Fix bugs to pass TPCDS cases ( #15194 )
2022-12-20 22:29:55 +08:00
5cf21fa7d1
[feature](planner) mark join to support subquery in disjunction ( #14579 )
...
Co-authored-by: Gabriel <gabrielleebuaa@gmail.com >
2022-12-20 15:22:43 +08:00
7c67fa8651
[Bug](pipeline) fix bug of right anti join error result in pipeline ( #15165 )
2022-12-19 19:28:44 +08:00
0732f31e5d
[Bug](pipeline) Fix bugs for scan node and join node ( #15164 )
...
* [Bug](pipeline) Fix bugs for scan node and join node
* update
2022-12-19 15:59:29 +08:00
874acdf68f
[vectorized](join) add try catch in create thread ( #15065 )
2022-12-16 19:55:09 +08:00
284a3351f4
[Refactor](exec) refactor the code of datasink eos logic ( #15009 )
2022-12-13 15:33:08 +08:00
68092fe514
[pipeline](NLJ) support nested loop join for pipeline ( #14966 )
2022-12-10 00:20:16 +08:00
9d36931038
[Refactor](NLJ) refactor the nested loop join node ( #14911 )
...
* [Refactor](NLJ) refactor the nested loop join node
* change the logic of alloc/release resource
2022-12-09 14:10:26 +08:00
0c817e6b3a
[Pipeline](hashjoin) Support hash join on pipeline engine ( #14898 )
2022-12-08 15:43:02 +08:00
8c0e13ab51
[improvement](profile) add detail memory counter for exec nodes ( #14806 )
...
* [improvement](profile) improve accuraccy of memory usage and add detail memory counter
* fix
2022-12-05 11:51:52 +08:00
176f519fa1
[enhancement](memtracker) Optimize exec node memory tracking ( #14711 )
2022-12-01 14:52:21 +08:00
b4d32a0c44
[fix](join) runtime filter shared from other instance wasn't be published ( #14717 )
2022-12-01 14:17:23 +08:00
6c70d794f6
[fix](bitmapfilter) fix core dump caused by bitmap filter ( #14702 )
2022-12-01 09:56:22 +08:00
7513c82431
[NLJoin](conjuncts) separate join conjuncts and general conjuncts ( #14608 )
2022-11-29 08:55:54 +08:00
78adecac1b
[enhancemennt](be)optimize mem usage in join and set node ( #14602 )
2022-11-27 13:38:49 +08:00
4728e75079
[feature](bitmap) Support in bitmap syntax and bitmap runtime filter ( #14340 )
...
1.Support in bitmap syntax, like 'where k1 in (select bitmap_column from tbl)';
2.Support bitmap runtime filter. Generate a bitmap filter using the right table bitmap and push it down to the left table storage layer for filtering.
2022-11-25 15:22:44 +08:00
9103ded1dd
[improvement](join)optimize sharing hash table for broadcast join ( #14371 )
...
This PR is to make sharing hash table for broadcast more robust:
Add a session variable to enable/disable this function.
Do not block the hash join node's close function.
Use shared pointer to share hash table and runtime filter in broadcast join nodes.
The Hash join node that doesn't need to build the hash table will close the right child without reading any data(the child will close the corresponding sender).
2022-11-24 21:06:44 +08:00
6c7f758ef7
[improvement](hashjoin) support partitioned hash table in hash join ( #14480 )
2022-11-24 14:16:47 +08:00
2c42f0a905
[refactor](decimalv3) Refine code for DecimalV3 ( #14394 )
2022-11-19 16:57:17 +08:00
1f326fc0d6
[enhancement](be)limit mem cost to 16m when pre serialize keys in agg node ( #14321 )
...
* [enhancement](be)limit mem cost to 16m when pre serialize keys in agg node
* use only one chunk memory when serializing keys in agg node
2022-11-18 12:31:52 +08:00
50bfd99b59
[feature](join) support nested loop semi/anti join ( #14227 )
2022-11-17 22:20:08 +08:00
d5af4f6558
[Neried](Profile) Add projection timer for neried ( #14286 )
2022-11-17 22:17:55 +08:00
6d2e6d85d3
[enhancement](be)release memory in Node's close() method ( #14258 )
...
* [enhancement](be)release memory in Node's close() method
* format code
2022-11-15 15:59:23 +08:00
215a4c6e02
[Bug](BHJ) Fix wrong result when use broadcast hash join for naaj ( #14253 )
2022-11-15 09:40:00 +08:00
7bb3792d51
[chore](build) Split the compliation units to build them in parallel ( #14232 )
2022-11-14 10:57:10 +08:00
139c4a77f1
[enhancement](be)close ExecNode ASAP to release resource earlier ( #14203 )
2022-11-14 09:41:35 +08:00
dd11d5c0a5
[enhancement](memory) Support try catch bad alloc ( #14135 )
2022-11-13 11:22:56 +08:00
035657c5a1
[typo](comment) Fix a lot of spell errors in be comments ( #14208 )
...
fix typos.
2022-11-12 16:06:15 +08:00
fe2944d56d
[Bug](nljoin) Keep compatibility for nljoin ( #14182 )
2022-11-11 15:54:55 +08:00
118a7dff07
[chore](build) Optimize the compilation time ( #14170 )
...
Currently, it takes too much time to build BE from source in workflow environments (P0/P1) which affects the efficiency of daily development.
We can measure the time by executing the following command.
time EXTRA_CXX_FLAGS='-O3' BUILD_TYPE=ASAN ./build.sh --be --fe --clean -j "$(nproc)"
This PR optimizes the compilation time by exploiting the following methods.
Reduce the codegen by removing some useless std::visit.
Disable the optimization for some template functions which are instantiated by std::visit conditionally (except for the RELEASE build).
2022-11-11 12:09:54 +08:00
