1d421a26d9
[bugfix](memory) merge block may allocate failed ( #19507 )
2023-05-11 10:42:47 +08:00
aabcab9dbe
[Improvement](runtime filter) Improve merge phase ( #18828 )
2023-04-26 21:01:20 +08:00
8e4710079d
[improvement](profile) Insert into add LoadChannel runtime profile ( #18908 )
...
TabletSink and LoadChannel in BE are M: N relationship,
Every once in a while LoadChannel will randomly return its own runtime profile to a TabletSink, so usually all LoadChannel runtime profiles are saved on each TabletSink, and the timeliness of the same LoadChannel profile saved on different TabletSinks is different, and each TabletSink will periodically send fe reports all the LoadChannel profiles saved by itself, and ensures to update the latest LoadChannel profile according to the timestamp.
2023-04-24 09:41:57 +08:00
3736530585
[refactor](query context) rename query fragments context to query context and make query context safe ( #18950 )
...
* [refactor](query context) rename query fragments context to query context and make query context safe
---------
Co-authored-by: yiguolei <yiguolei@gmail.com >
2023-04-23 22:53:56 +08:00
293e115536
[Improvement](bloom filter) initialize bloom filter with adaptive size ( #18785 )
2023-04-20 10:06:40 +08:00
e412dd12e8
[chore](build) Use include-what-you-use to optimize includes (PART II) ( #18761 )
...
Currently, there are some useless includes in the codebase. We can use a tool named include-what-you-use to optimize these includes. By using a strict include-what-you-use policy, we can get lots of benefits from it.
2023-04-19 23:11:48 +08:00
79c446c89f
[enhancement](exception) Column filter/replicate supports exception safety ( #18503 )
2023-04-18 19:23:09 +08:00
e5793249cd
[opt](hashtable) Modify default filled strategy to 75% ( #18242 )
2023-03-31 09:28:11 +08:00
78abb40fdc
[improvement](string) throw exception instead of log fatal if string column exceed total size limit ( #17989 )
...
Throw exception instead of log fatal if string column exceed total size limit, so that we can catch it and let query fail, instead of causing be exit.
2023-03-27 08:55:26 +08:00
7d91114304
[fix](join) fix wrong result of null aware left anti join ( #17752 )
2023-03-14 09:35:46 +08:00
93a865c3e8
[improvement](join) Avoid reading from left child while hash table is empty(right join) ( #17655 )
...
When the right (build) side is empty in a right outer join, there is no need to read data from the left child.
2023-03-13 09:03:17 +08:00
a1c0054b4c
[fix](memory) fix memory GC details and join probe catch bad_alloc ( #16989 )
...
Fix Redhat 4.x OS /proc/meminfo has no MemAvailable, disable MemAvailable to control memory.
vm_rss_str and mem_available_str recorded when gc is triggered, to avoid memory changes during gc and cause inaccurate logs.
join probe catch bad_alloc, this may alloc 64G memory at a time, avoid OOM.
Modify document doris_be_all_segments_num and doris_be_all_rowsets_num names.
2023-02-23 08:33:30 +08:00
f6a20f844b
[fix](hashjoin) join produce blocks with rows larger than batch size: handle join with other conjuncts ( #16402 )
2023-02-08 14:26:35 +08:00
696c6ffcc5
[fix](join) crash caused by canceling query ( #16311 )
...
If the query was canceled,
the status in shared context may be `OK` with other fields not set.
2023-02-02 09:55:37 +08:00
bf16228851
[fix](hashjoin) join produce blocks with rows larger than batch size ( #16166 )
...
* [fix](hashjoin) join produce blocks with rows larger than batch size
* fix
2023-02-01 16:02:31 +08:00
79ad74637d
[refactor](remove expr) remove non vectorized Expr and ExprContext related codes ( #16136 )
2023-01-24 10:45:35 +08:00
9f106161a7
[Bug](join) Fix null aware anti join error in fuzzy mode ( #15987 )
2023-01-17 11:32:16 +08:00
97fcad76f8
[enhancement](memtracker) Improve readability ( #15716 )
2023-01-16 16:30:35 +08:00
9468711f9f
[Bug](join) fix bug null aware left anti join not correct result ( #15841 )
2023-01-13 10:18:05 +08:00
9c36278c4a
[improvement](pipeline) Support sharing hash table for broadcast join ( #15628 )
2023-01-06 15:11:28 +08:00
05d72e8919
[fix](join) fix anti join incorrectly outputs null values ( #15567 )
2023-01-06 09:55:48 +08:00
5ff5b8fc98
[feature](mark join) Support mark join for hash join node ( #15569 )
...
* [feature](mark join) Support mark join for hash join node
2023-01-05 09:32:26 +08:00
10be583e52
[chore](pipeline) optimize profile information ( #15433 )
2022-12-30 09:56:33 +08:00
06f71f2bca
[pipeline](fix) Fix bugs to pass all regression cases ( #15306 )
...
* [pipeline](fix) Fix bugs to pass all regression cases
* update
* update
2022-12-23 22:17:50 +08:00
b085ff49f0
[refactor](non-vec) delete non-vec data sink ( #15283 )
...
* [refactor](non-vec) delete non-vec data sink
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2022-12-23 14:10:47 +08:00
8ecf69b09b
[pipeline](regression) nested loop join test get error result in pipeline engine and refactor the code for need more input data ( #15208 )
2022-12-21 19:03:51 +08:00
af54299b26
[Pipeline](projection) Support projection on pipeline engine ( #15220 )
2022-12-21 15:47:29 +08:00
732417258c
[Bug](pipeline) Fix bugs to pass TPCDS cases ( #15194 )
2022-12-20 22:29:55 +08:00
7c67fa8651
[Bug](pipeline) fix bug of right anti join error result in pipeline ( #15165 )
2022-12-19 19:28:44 +08:00
0732f31e5d
[Bug](pipeline) Fix bugs for scan node and join node ( #15164 )
...
* [Bug](pipeline) Fix bugs for scan node and join node
* update
2022-12-19 15:59:29 +08:00
0c817e6b3a
[Pipeline](hashjoin) Support hash join on pipeline engine ( #14898 )
2022-12-08 15:43:02 +08:00
8c0e13ab51
[improvement](profile) add detail memory counter for exec nodes ( #14806 )
...
* [improvement](profile) improve accuraccy of memory usage and add detail memory counter
* fix
2022-12-05 11:51:52 +08:00
176f519fa1
[enhancement](memtracker) Optimize exec node memory tracking ( #14711 )
2022-12-01 14:52:21 +08:00
b4d32a0c44
[fix](join) runtime filter shared from other instance wasn't be published ( #14717 )
2022-12-01 14:17:23 +08:00
7513c82431
[NLJoin](conjuncts) separate join conjuncts and general conjuncts ( #14608 )
2022-11-29 08:55:54 +08:00
78adecac1b
[enhancemennt](be)optimize mem usage in join and set node ( #14602 )
2022-11-27 13:38:49 +08:00
9103ded1dd
[improvement](join)optimize sharing hash table for broadcast join ( #14371 )
...
This PR is to make sharing hash table for broadcast more robust:
Add a session variable to enable/disable this function.
Do not block the hash join node's close function.
Use shared pointer to share hash table and runtime filter in broadcast join nodes.
The Hash join node that doesn't need to build the hash table will close the right child without reading any data(the child will close the corresponding sender).
2022-11-24 21:06:44 +08:00
6c7f758ef7
[improvement](hashjoin) support partitioned hash table in hash join ( #14480 )
2022-11-24 14:16:47 +08:00
2c42f0a905
[refactor](decimalv3) Refine code for DecimalV3 ( #14394 )
2022-11-19 16:57:17 +08:00
50bfd99b59
[feature](join) support nested loop semi/anti join ( #14227 )
2022-11-17 22:20:08 +08:00
6d2e6d85d3
[enhancement](be)release memory in Node's close() method ( #14258 )
...
* [enhancement](be)release memory in Node's close() method
* format code
2022-11-15 15:59:23 +08:00
215a4c6e02
[Bug](BHJ) Fix wrong result when use broadcast hash join for naaj ( #14253 )
2022-11-15 09:40:00 +08:00
7bb3792d51
[chore](build) Split the compliation units to build them in parallel ( #14232 )
2022-11-14 10:57:10 +08:00
139c4a77f1
[enhancement](be)close ExecNode ASAP to release resource earlier ( #14203 )
2022-11-14 09:41:35 +08:00
dd11d5c0a5
[enhancement](memory) Support try catch bad alloc ( #14135 )
2022-11-13 11:22:56 +08:00
035657c5a1
[typo](comment) Fix a lot of spell errors in be comments ( #14208 )
...
fix typos.
2022-11-12 16:06:15 +08:00
fe2944d56d
[Bug](nljoin) Keep compatibility for nljoin ( #14182 )
2022-11-11 15:54:55 +08:00
118a7dff07
[chore](build) Optimize the compilation time ( #14170 )
...
Currently, it takes too much time to build BE from source in workflow environments (P0/P1) which affects the efficiency of daily development.
We can measure the time by executing the following command.
time EXTRA_CXX_FLAGS='-O3' BUILD_TYPE=ASAN ./build.sh --be --fe --clean -j "$(nproc)"
This PR optimizes the compilation time by exploiting the following methods.
Reduce the codegen by removing some useless std::visit.
Disable the optimization for some template functions which are instantiated by std::visit conditionally (except for the RELEASE build).
2022-11-11 12:09:54 +08:00
1ef85ae1f2
[Improvement](join) Support nested loop outer join ( #13965 )
2022-11-10 19:50:46 +08:00
10df61b5bf
[improvement](join) Share hash table in fragments for broadcast join ( #13921 )
2022-11-10 09:48:34 +08:00
