9434ee5710
[fix](load) fix memtracking orphan too large ( #28600 )
2023-12-19 12:41:19 +08:00
66fbb22ad7
[fix](group commit) Fix some wal problems on group commit ( #28554 )
2023-12-19 09:51:03 +08:00
469edbdd3d
[feature](executor)make scan task wait timeout config #28467
2023-12-16 11:36:15 +08:00
82a91380e6
[enhancement](compaction) Add support for limiting low priority compaction scheduling ( #27648 )
2023-12-14 18:31:23 +08:00
e6e8632167
[improvement](merge-on-write) Optimize publish when there are missing versions ( #28012 )
...
1. Do not retry publishing on be When there are too many missing versions, just
add to async publish task.
2. To reduce memory consumption, clean up the tasks when there are too many
async publish tasks.
2023-12-13 16:59:25 +08:00
b901800963
[Fix] Support BE log sys_log_level modify to take effect dynamically (apache#26060) ( #28203 )
2023-12-13 11:08:48 +08:00
45b2dbab6a
[improve](group commit) Group commit support max filter ratio when rows is less than value in config ( #28139 )
2023-12-12 16:33:36 +08:00
e49ed3d885
[regression test](memtable) add case for aggregation memtable ( #28056 )
...
1. create aggregation table
2. insert some data
3. drop the table and create again
4. modify some parameters for some branch
5. insert some data
6. change the parameters back to its default
2023-12-12 11:14:59 +08:00
7fba3fcb91
[pipelineX](improvement) block local shuffle sink by mem usage ( #28224 )
2023-12-11 21:25:31 +08:00
593cc92501
[chore] Change default max segment size to 1GB ( #28201 )
2023-12-11 14:30:57 +08:00
59ec3da899
open workload group in PR pipeline ( #27744 )
2023-12-08 11:56:03 +08:00
394b420180
[Update](inverted index) use session variable for inverted index try query threshold ( #28052 )
...
* [Update](inverted index) use session variable for inverted index try query threshold
* remove unused config
* update clucene
2023-12-07 17:54:44 +08:00
a27c068a9d
[improve](move-memtable) make StreamWait time configurable ( #28086 )
2023-12-07 17:27:43 +08:00
84a651d976
[improve](load) rewrite memtable memory limiter rules ( #27759 )
2023-12-07 17:26:26 +08:00
48935c14e2
[Improvement](variant) limit the column size on tablet schema ( #27399 ) ( #27785 )
...
1. limit the column count to default 2048
2. fix get_inverted_index return nullptr when variant's unique id is -1, using it's parent unique id instead
3. avoid add same path subcolumn duplicately in tablet schema
4. make extracted column unique id -1
2023-12-04 14:47:36 +08:00
db8e56b9f2
[improve](move-memtable) increase open load stream timeout ( #26909 )
2023-11-30 15:27:29 +08:00
112ae59aa4
[fix](move-memtable) add timeout for load stream close wait ( #27439 )
2023-11-30 12:00:06 +08:00
d96e2dfefb
[feature-wip](arrow-flight)(step5) Support JDBC and PreparedStatement and Fix Bug ( #27661 )
2023-11-29 21:17:20 +08:00
9daa7dc6b5
[refactor](http) disable snapshot and get_log_file api ( #27724 )
...
Disable 2 http api by default:
1. BE's `/api/snapshot`
2. FE's `/get_log_file`
2023-11-29 16:11:51 +08:00
7398c3daf1
[Feature-Variant](Variant Type) support variant type query and index ( #27676 )
2023-11-29 10:37:28 +08:00
0a8e3d2199
[enhance](PrefetchReader) Make the prefetch timeout one config ( #27371 )
2023-11-24 10:24:15 +08:00
df628e1538
[chore](merge-on-write) disable rowid conversion check for mow table by default ( #27482 )
2023-11-23 23:39:01 +08:00
2ea33518b0
[Opt](load) use batching to optimize auto partition ( #26915 )
...
use batching to optimize auto partition
2023-11-23 19:12:28 +08:00
7a75f8c380
[improve](move-memtable) set brpc streaming params in config ( #27442 )
2023-11-23 14:14:43 +08:00
1cd1c58eee
[Feature](group commit) move group_commit_interval_ms from be.conf to table property ( #27116 )
2023-11-21 20:50:02 +08:00
be7273da83
[refactor](executor)Refactor workload meta update to be #26710
2023-11-18 11:19:38 +08:00
5d548935e0
[improvement](insert) support schema change and decommission for group commit ( #26359 )
2023-11-17 21:41:38 +08:00
54989175fb
[case] Load json data with enable_simdjson_reader=false ( #26601 )
2023-11-16 14:40:59 +08:00
dbac12bae8
[fix](memory)Modify the default conf values of mem_limit and cache_last_version_interval_second ( #26945 )
...
mem_limit from 80% to 90%
cache_last_version_interval_second from 900 to 30
2023-11-15 14:02:58 +08:00
89215306d3
[improve](load) add switch for vertical segment writer ( #26996 )
2023-11-15 08:19:12 +08:00
c0fda8c5c2
[improve](group commit) Add a swicth to wait internal group commit lo… ( #26734 )
...
* [improve](group commit) Add a swicth to make internal group commit load finish
* modify group commit tvf plan
2023-11-13 10:35:35 +08:00
8b33b0c4a4
[Fix](row store) cache invalidate key should not include sequence column ( #26771 )
2023-11-11 01:30:32 -06:00
d767804815
[feature](merge-cloud) Decouple rowset id generator and local rowsets gc implementation ( #25921 )
2023-11-10 10:07:02 +08:00
a5565f68b2
[Refactor](opentelemetry) Remove opentelemetry ( #26605 )
2023-11-09 18:05:34 +08:00
5f62a4462d
[Enhancement](wal) Add wal space back pressure ( #26483 )
2023-11-09 12:29:05 +08:00
33e46ee13d
[enhancement](config) enable single_replica_load by default in BE ( #26619 )
...
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com >
2023-11-09 12:14:37 +08:00
58bf79f79e
[fix](move-memtable) pass load stream num to backends ( #26198 )
2023-11-08 16:16:33 +08:00
6637f9c15f
Add enable_cgroup_cpu_soft_limit ( #26510 )
2023-11-08 15:52:13 +08:00
44b51bf0b9
[Feature](Variant) support variant load ( #26572 )
2023-11-08 00:37:57 -06:00
2cc68381ec
[feature](binlog) Add ingest_binlog/http_get_snapshot limit download speed && Add async ingest_binlog ( #26323 )
2023-11-06 11:14:44 +08:00
be7a10162a
[enhance](S3) Add timeout for s3 buffer allocation and corresponding observability ( #26125 )
2023-11-01 17:55:11 +08:00
387e33fa34
[enhancement](group commit)Add group commit block queues memory back pressure ( #26045 )
2023-11-01 16:29:45 +08:00
6a85f46ff3
[refactor](move-memtable) rename open_stream_sink rpc to open_load_stream ( #25883 )
2023-10-29 10:07:14 +08:00
46d40b1952
[refactor](executor)Remove empty group logic #26005
2023-10-27 14:24:41 +08:00
d0cd535cb9
[improvement](insert) refactor group commit stream load ( #25560 )
2023-10-20 13:27:30 +08:00
9a675fcdfc
[chore](be) Add default timezone files ( #25097 )
2023-10-20 13:12:24 +08:00
b74836050a
[chore](config) turnoff fuzzy for enable_simdjson_reader ( #25521 )
2023-10-17 18:42:11 +08:00
59ebbb351e
[feature](merge-cloud) Enable write into cache when uploading file to s3 using s3 file writer ( #24364 )
2023-10-16 21:31:02 +08:00
eaf5febc97
[enhancement](cooldown) Improve cooldown logs ( #25432 )
2023-10-16 20:17:00 +08:00
283bd59eba
[improvement](scanner) Remove the predicate that is always true for the segment ( #25366 )
...
By utilizing the zonemap index of the segment, we can ascertain if a predicate is always true. For example, if the segment’s maximum value is 100 and the predicate is col < 101, then this predicate is always true for this segment.
2023-10-13 15:25:38 +08:00
