66fbb22ad7
[fix](group commit) Fix some wal problems on group commit ( #28554 )
2023-12-19 09:51:03 +08:00
45b2dbab6a
[improve](group commit) Group commit support max filter ratio when rows is less than value in config ( #28139 )
2023-12-12 16:33:36 +08:00
d4f89389e3
[improve](group commit) Group commit support skip wal ( #27957 )
2023-12-11 19:38:32 +08:00
cff1de29ce
[fix](group commit) Fix group commit memory calculation ( #28242 )
2023-12-11 17:05:26 +08:00
1e5ff40e17
[refactor](group commit) remove future block ( #27720 )
...
Co-authored-by: huanghaibin <284824253@qq.com >
2023-12-11 08:41:51 +08:00
13b26ee920
[Fix](core) Fix wal space back pressure core and add regression test ( #27311 )
2023-11-27 15:10:26 +08:00
b457856bd2
[chore](be) remove bthread scanner related codes ( #27417 )
2023-11-23 15:18:49 +08:00
1cd1c58eee
[Feature](group commit) move group_commit_interval_ms from be.conf to table property ( #27116 )
2023-11-21 20:50:02 +08:00
5d548935e0
[improvement](insert) support schema change and decommission for group commit ( #26359 )
2023-11-17 21:41:38 +08:00
c0fda8c5c2
[improve](group commit) Add a swicth to wait internal group commit lo… ( #26734 )
...
* [improve](group commit) Add a swicth to make internal group commit load finish
* modify group commit tvf plan
2023-11-13 10:35:35 +08:00
b19f275714
[improvement](insert) refactor group commit insert into ( #25795 )
2023-11-03 12:02:40 +08:00
387e33fa34
[enhancement](group commit)Add group commit block queues memory back pressure ( #26045 )
2023-11-01 16:29:45 +08:00
e20cab64f4
[improvement](scan) avoid too many scanners for file scan node ( #25727 )
...
In previous, when using file scan node(eq, querying hive table), the max number of scanner for each scan node
will be the `doris_scanner_thread_pool_thread_num`(default is 48).
And if the query parallelism is N, the total number of scanner would be 48 * N, which is too many.
In this PR, I change the logic, the max number of scanner for each scan node
will be the `doris_scanner_thread_pool_thread_num / query parallelism`. So that the total number of scanners
will be up to `doris_scanner_thread_pool_thread_num`.
Reduce the number of scanner can significantly reduce the memory usage of query.
2023-10-29 17:41:31 +08:00
fded8ba824
[improvement](insert) Modify insert into table_id(xxx) to doris_inter… ( #25662 )
2023-10-20 19:31:37 +08:00
d0cd535cb9
[improvement](insert) refactor group commit stream load ( #25560 )
2023-10-20 13:27:30 +08:00
f1a5e393c7
[feature](insert) Support group commit insert use new syntax like insert into table_id(xxx) ( #25484 )
2023-10-17 09:23:09 +08:00
1073ef22f3
[fix](insert) improve group_commit related tests ( #25319 )
2023-10-12 21:19:29 +08:00
4ce5213b1c
[fix](insert) Fix test_group_commit_stream_load and add more regression in test_group_commit_http_stream ( #24954 )
2023-10-03 20:56:24 +08:00
642e5cdb69
[Fix](Status) Make Status [[nodiscard]] and handle returned Status correctly ( #23395 )
2023-09-29 22:38:52 +08:00
55d1090137
[feature](insert) Support group commit stream load ( #24304 )
2023-09-26 20:57:02 +08:00
082bcd820b
[feature](insert) Support wal for group commit insert ( #23053 )
2023-09-26 14:46:24 +08:00
c7ae2a7d22
[Refactor & Bugfix](static variables) move some static vairables to exec_env ( #24029 )
2023-09-13 09:27:03 +08:00
82dc970916
[feature](insert) Support group commit insert ( #22829 )
2023-09-08 15:51:03 +08:00
