45b2dbab6a
[improve](group commit) Group commit support max filter ratio when rows is less than value in config ( #28139 )
2023-12-12 16:33:36 +08:00
d4f89389e3
[improve](group commit) Group commit support skip wal ( #27957 )
2023-12-11 19:38:32 +08:00
877935442f
[feature](pipelineX)use markFragments instead of markInstances in pipelineX ( #27829 )
2023-12-11 17:59:53 +08:00
cff1de29ce
[fix](group commit) Fix group commit memory calculation ( #28242 )
2023-12-11 17:05:26 +08:00
1e5ff40e17
[refactor](group commit) remove future block ( #27720 )
...
Co-authored-by: huanghaibin <284824253@qq.com >
2023-12-11 08:41:51 +08:00
4e86f9bab5
[improve](move-memtable) include and check offset when append data ( #28159 )
2023-12-09 16:21:36 +08:00
6da36e1077
[feature](merge-cloud) Refactor write path code by abstract base class ( #26537 )
...
Refactor write path code by abstract base class. Whether to use `StorageEngine` or `CloudStorageEngine` will be determined during compilation instead of runtime `config::cloud_mode` to avoid unexpected null pointer or undefined behavior issues caused by merging code.
Class that depend on `StorageEngine` but are shared by the cloud mode need to have an abstract base class. Common code should be extracted into the base class, while the code that depends on `StorageEngine` should be implemented in a `StorageEngine` mix-in class of the base class.
2023-12-08 14:50:36 +08:00
9c63dfd692
[improve](move-memtable) support variant in move-memtable ( #28084 )
...
* [improve](move-memtable) support variant in move-memtable
Pass the flush schema to the destination end of the sink.
2023-12-07 16:41:38 +08:00
c5f00fa746
[fix](move-memtable) temporary disable load stream profile ( #28029 )
2023-12-07 12:05:29 +08:00
bd808ae3b4
[cleanup](load) remove unused LoadChannel::get_tablets_channels ( #27981 )
2023-12-06 20:50:26 +08:00
3595f21405
[improvement](executor)clear unused cgroup path ( #27798 )
...
* clear unused cgroup path
* use C++ api
* add gcc header
2023-12-05 14:18:23 +08:00
54fe1a166b
[Refactor](scan) refactor scan scheduler to improve performance ( #27948 )
...
* [Refactor](scan) refactor scan scheduler to improve performance
* fix pipeline x core
2023-12-05 13:03:16 +08:00
17016b9797
[improvement](decimal) use new way for decimal arithmetic precision promotion ( #27787 )
...
* [DNM](decimal) use new way for decimal arithmetic precision promotion
* [improvement](decimal) [DNM](decimal) use new way for decimal arithmetic precision promotion
1. [DNM](decimal) use new way for decimal arithmetic precision promotion
2. throw exception if it overflows for decimal arithmetics
3. throw exception if it overflows when casting among number types
* fix compile error of gcc
* improvement
---------
Co-authored-by: morrySnow <morrysnow@126.com >
2023-12-05 12:54:40 +08:00
75d0beb8cc
[fix](move-memtable) only report load stream profile in the end ( #27983 )
2023-12-05 11:30:54 +08:00
bd9db7423b
[fix](move-memtable) free resources before storage engine stop ( #27980 )
2023-12-05 11:15:05 +08:00
e3d2425d47
[Improvement](join) remove insert_indices_from_join and special judge for -1 ( #27779 )
...
remove insert_indices_from_join and special judge for -1
2023-12-04 11:03:22 +08:00
be30bd1e40
[improvement](spinlock) remove some potential bad spinlock usage ( #27904 )
...
* [improvement](spinlock) remove some potential spinlock usage
---------
Co-authored-by: yiguolei <yiguolei@gmail.com >
2023-12-02 20:33:54 +08:00
421ab56c3e
[pipelineX](improvement) Support local shuffle for join and agg ( #27852 )
2023-12-02 20:17:18 +08:00
a1a75fcfbd
[fix](runtime filter) Fix extremely high CPU usage caused by rf merge #27894
2023-12-02 07:40:52 +08:00
1706699e7e
[fix](multi-catalog)support the max compute partition prune ( #27154 )
...
1. max compute partition prune,
we just support filter mc partitions by '='，it can filter just one partition
to support multiple partition filter and range operator('>','<', '>='..), the partition prune should be supported.
2. add max compute row count cache and partitionValues cache
3. add max compute regression case
2023-12-01 22:28:26 +08:00
0b7becd4b7
[fix](executor)Fix memtracker not set to task group #27699
2023-11-30 22:35:51 +08:00
5a4948f0f9
[fix](load) fix DataSink prepared check in PlanFragmentExecutor ( #27735 )
2023-11-30 15:24:04 +08:00
34e53acaea
[pipelineX](fix) Fix local exchange on pipelineX engine ( #27763 )
2023-11-30 11:16:20 +08:00
f1e9e6dba8
[fix](pipelineX) make RuntimeFilterTimerQueue graceful exit ( #27653 )
...
make RuntimeFilterTimerQueue graceful exit
2023-11-29 18:53:13 +08:00
d9d5468621
[feature](audit-log) add audit-log in insert into ( #27641 )
2023-11-29 15:01:57 +08:00
b93dd1d5f7
[enhancement](load) improve error msg for load when cancelled by mem gc ( #26809 )
...
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com >
2023-11-28 17:36:11 +08:00
f565f60bc3
[refactor](standard)BE:Initialize pointer variables in the class to nullptr by default ( #27587 )
2023-11-28 13:02:30 +08:00
b48c40ed31
Make blockschduler first stop then delete ( #27645 )
2023-11-28 10:09:15 +08:00
13b26ee920
[Fix](core) Fix wal space back pressure core and add regression test ( #27311 )
2023-11-27 15:10:26 +08:00
3838b6fbae
[refine](pipelineX) refine some code in pipelineX ( #27472 )
2023-11-27 11:04:16 +08:00
d0fea8db27
[chore][log] Opt log, revert some log introduced by #25739 ( #26365 )
2023-11-27 10:48:02 +08:00
f3204ec18c
[fix](memory) Fix thread_context check failed: thread_mem_tracker()->label() == "Orphan" #27468
2023-11-24 20:57:41 +08:00
89d5efac46
[pipelineX](dependency) Use dependency to replace ready_to_execute ( #27528 )
2023-11-24 16:03:14 +08:00
2ea33518b0
[Opt](load) use batching to optimize auto partition ( #26915 )
...
use batching to optimize auto partition
2023-11-23 19:12:28 +08:00
b457856bd2
[chore](be) remove bthread scanner related codes ( #27417 )
2023-11-23 15:18:49 +08:00
4fc638c98c
[fix](memory) Fix make_top_consumption_snapshots heap-use-after-free #27434
2023-11-23 13:59:53 +08:00
75d1594d11
[pipelineX](minor) remove unused code ( #27446 )
2023-11-23 11:53:06 +08:00
b79f5d77f1
[improve](move-memtable) improve logging messages ( #27443 )
2023-11-23 11:46:29 +08:00
1b3512d942
[pipelineX](bug) Fix cancel timeout ( #27396 )
2023-11-22 22:31:34 +08:00
5442e8d1fc
[pipelineX](dependency) split different dependencies ( #27366 )
2023-11-22 12:50:39 +08:00
1cd1c58eee
[Feature](group commit) move group_commit_interval_ms from be.conf to table property ( #27116 )
2023-11-21 20:50:02 +08:00
3e8177bbbd
Revert "[improvement](routine-load) add routine load rows check ( #25818 )" ( #27304 )
...
This reverts commit 5e3277e8fbc92f08344bc86e9b2c74033a8bbc01.
2023-11-21 11:28:05 +08:00
70e070182f
[feature](executor)Make workload group property not required ( #27229 )
...
* Make workload group property not required
* remove useless UT
2023-11-19 17:01:51 +08:00
7b817197b2
[pipelineX](sink) simplify result sink dependency ( #27226 )
2023-11-19 12:42:10 +08:00
be7273da83
[refactor](executor)Refactor workload meta update to be #26710
2023-11-18 11:19:38 +08:00
b1eef30b49
[pipelineX](dependency) Wake up task by dependencies ( #26879 )
...
---------
Co-authored-by: Mryange <2319153948@qq.com >
2023-11-18 03:20:24 +08:00
5d548935e0
[improvement](insert) support schema change and decommission for group commit ( #26359 )
2023-11-17 21:41:38 +08:00
0a1a6cf02f
[fix](topn) add defensive code in topn opt to avoid crash due to column not in tablet schema
2023-11-17 21:14:10 +08:00
3ad865fef9
[refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits ( #26191 )
2023-11-15 21:34:49 +08:00
30d1e6036c
[feature](runtime filter) New session variable runtime_filter_wait_infinitely ( #26888 )
...
New session variable: runtime_filter_wait_infinitely. If set runtime_filter_wait_infinitely = true, consumer of rf will wait on receiving until query is timeout.
2023-11-14 21:05:59 +08:00
