54989175fb
[case] Load json data with enable_simdjson_reader=false ( #26601 )
2023-11-16 14:40:59 +08:00
dbac12bae8
[fix](memory)Modify the default conf values of mem_limit and cache_last_version_interval_second ( #26945 )
...
mem_limit from 80% to 90%
cache_last_version_interval_second from 900 to 30
2023-11-15 14:02:58 +08:00
89215306d3
[improve](load) add switch for vertical segment writer ( #26996 )
2023-11-15 08:19:12 +08:00
c0fda8c5c2
[improve](group commit) Add a swicth to wait internal group commit lo… ( #26734 )
...
* [improve](group commit) Add a swicth to make internal group commit load finish
* modify group commit tvf plan
2023-11-13 10:35:35 +08:00
8b33b0c4a4
[Fix](row store) cache invalidate key should not include sequence column ( #26771 )
2023-11-11 01:30:32 -06:00
d767804815
[feature](merge-cloud) Decouple rowset id generator and local rowsets gc implementation ( #25921 )
2023-11-10 10:07:02 +08:00
a5565f68b2
[Refactor](opentelemetry) Remove opentelemetry ( #26605 )
2023-11-09 18:05:34 +08:00
5f62a4462d
[Enhancement](wal) Add wal space back pressure ( #26483 )
2023-11-09 12:29:05 +08:00
33e46ee13d
[enhancement](config) enable single_replica_load by default in BE ( #26619 )
...
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com >
2023-11-09 12:14:37 +08:00
58bf79f79e
[fix](move-memtable) pass load stream num to backends ( #26198 )
2023-11-08 16:16:33 +08:00
6637f9c15f
Add enable_cgroup_cpu_soft_limit ( #26510 )
2023-11-08 15:52:13 +08:00
44b51bf0b9
[Feature](Variant) support variant load ( #26572 )
2023-11-08 00:37:57 -06:00
2cc68381ec
[feature](binlog) Add ingest_binlog/http_get_snapshot limit download speed && Add async ingest_binlog ( #26323 )
2023-11-06 11:14:44 +08:00
be7a10162a
[enhance](S3) Add timeout for s3 buffer allocation and corresponding observability ( #26125 )
2023-11-01 17:55:11 +08:00
387e33fa34
[enhancement](group commit)Add group commit block queues memory back pressure ( #26045 )
2023-11-01 16:29:45 +08:00
6a85f46ff3
[refactor](move-memtable) rename open_stream_sink rpc to open_load_stream ( #25883 )
2023-10-29 10:07:14 +08:00
46d40b1952
[refactor](executor)Remove empty group logic #26005
2023-10-27 14:24:41 +08:00
d0cd535cb9
[improvement](insert) refactor group commit stream load ( #25560 )
2023-10-20 13:27:30 +08:00
9a675fcdfc
[chore](be) Add default timezone files ( #25097 )
2023-10-20 13:12:24 +08:00
b74836050a
[chore](config) turnoff fuzzy for enable_simdjson_reader ( #25521 )
2023-10-17 18:42:11 +08:00
59ebbb351e
[feature](merge-cloud) Enable write into cache when uploading file to s3 using s3 file writer ( #24364 )
2023-10-16 21:31:02 +08:00
eaf5febc97
[enhancement](cooldown) Improve cooldown logs ( #25432 )
2023-10-16 20:17:00 +08:00
283bd59eba
[improvement](scanner) Remove the predicate that is always true for the segment ( #25366 )
...
By utilizing the zonemap index of the segment, we can ascertain if a predicate is always true. For example, if the segment’s maximum value is 100 and the predicate is col < 101, then this predicate is always true for this segment.
2023-10-13 15:25:38 +08:00
9cc0e9526a
[enhancement](merge-on-write) consider version count on size-based cu compaction policy ( #25352 )
2023-10-13 14:52:21 +08:00
1073ef22f3
[fix](insert) improve group_commit related tests ( #25319 )
2023-10-12 21:19:29 +08:00
022762d5f0
[fix](memory) Fix work load group GC and add logs to locate slow GC #24975
...
Fix work load group GC, add cancel load and add logs.
Unify the format and change all to lowercase of GC logs, avoid unnecessary trouble when grep or less
Add logs to help locate the cause of slow GC.
2023-10-12 10:33:56 +08:00
e41b03e530
[Fix](multi-catalog) delete hdfs hedged configs at BE side. ( #25094 )
...
Issue Number: close #25093
We can set hdfs hedged configs when creating catalog, just like this:
```
CREATE CATALOG `test_ctl` PROPERTIES (
...
"dfs.client.hedged.read.threadpool.size" = "128",
"dfs.client.hedged.read.threshold.millis" = "500",
...
);
```
It is redundant to set these configs at BE side, and it will brings an occasional bug at #25093 .
2023-10-11 23:25:30 +08:00
7b2ff38401
query cpu hard limit based on doris scheduler ( #24844 )
2023-10-07 12:03:07 +08:00
db6c16058a
[improve](move-memtable) always share load streams ( #24763 )
2023-10-05 22:09:59 +08:00
642e5cdb69
[Fix](Status) Make Status [[nodiscard]] and handle returned Status correctly ( #23395 )
2023-09-29 22:38:52 +08:00
864a0f9bcb
[opt](pipeline) Make pipeline fragment context send_report asynchronized ( #23142 )
2023-09-28 17:55:53 +08:00
082bcd820b
[feature](insert) Support wal for group commit insert ( #23053 )
2023-09-26 14:46:24 +08:00
8679095e5c
[feature](debug) support debug point used in debug code ( #24502 )
2023-09-25 17:56:12 +08:00
c9b2f4cb92
[workload](pipeline) Add cgroup cpu controller ( #24052 )
2023-09-21 21:49:33 +08:00
8eb14eec7c
[enhancement](baddisk) record bad disk in be_custom.conf to handle ( #24639 )
2023-09-21 18:31:58 +08:00
36c9366a8b
[improve](tablet schema) add config to modify tablet schema recycle interval ( #24602 )
2023-09-21 15:57:11 +08:00
4eb09ce1b2
[enhancement](config) do not abort when a disk is broken ( #24692 )
2023-09-21 15:21:42 +08:00
00b994fea2
[chore](exception) Add config item 'exit_on_exception' ( #24529 )
2023-09-21 14:51:05 +08:00
f2f591e280
[fix](memory) Optimize memory exceed limit logs ( #22655 )
...
After memory exceeds the limit, print the top 15 task trackers with the largest memory.
After memory exceeds the limit, more detailed GC logs in stages.
fix large memory check.
2023-09-21 10:38:17 +08:00
1405b7ca82
[improve](scan) support lower the thread priority of scan thread ( #24526 )
...
The configuration item is used to lower the priority of the scanner thread,
typically employed to ensure CPU scheduling for write operations.
2023-09-20 17:00:24 +08:00
fc12362a6d
[feature-wip](arrow-flight)(step2) FE support Arrow Flight server ( #24314 )
...
This is a POC, the design documentation will be updated soon
2023-09-20 14:42:54 +08:00
a2e29d171a
[enhancement](be-meta) sync rocksdb by default to protect data ( #24571 )
...
If performance of user's disks is low, users can change the config
to false, this way users know what would happen if a kernel panic.
2023-09-20 11:41:26 +08:00
7fd72351f9
[fix](agg) windown_funnel compatibility issue with multi backends ( #24385 )
2023-09-15 17:22:47 +08:00
927de33166
[config](log) disable StreamLoad log default and enable in regression pipeline ( #24354 )
...
disable StreamLoad log default and enable in regression pipeline
2023-09-14 20:47:26 +08:00
86aa3802cf
[log](config) set streamload record default to enable
2023-09-13 16:32:30 +08:00
563c3f75ff
[feature](move-memtable) share delta writer v2 among sinks ( #24066 )
2023-09-13 14:39:29 +08:00
d8ef9dda59
[feature](merge-cloud) Rewrite FS interface ( #23953 )
2023-09-12 19:20:25 +08:00
71645a391c
[debug](FileCache) fail over to remote file reader if local cache failed ( #24097 )
...
Fail over to remote file reader even if local file cache failed. This operation can increase the robustness of file cache.
2023-09-10 12:26:17 +08:00
82dc970916
[feature](insert) Support group commit insert ( #22829 )
2023-09-08 15:51:03 +08:00
2f8b075b71
[improvement](bitmap) support version for ser/deser of bitmap ( #23959 )
2023-09-07 09:55:29 +08:00
