9eef393e2a
[pick]support cgroup v2 ( #42465 )
...
## Proposed changes
pick #39991 #39374 #36663
2024-10-25 20:13:27 +08:00
d659750fd9
[pick](Serde-2.1) fix variant serde may lost num_rows when subcolumns empty ( #41438 )
...
serialization object with empty subcolumns may lost num_rows, so need to
record num_rows and set back num_rows in serdes
backport #38413
2024-09-29 09:45:37 +08:00
6ceb574aa0
[branch-2.1]Pick IO limit/workload group usage table ( #39839 )
2024-08-23 18:51:47 +08:00
0934fbee7e
[improvement](query) prefer to chose tablet on alive disk #39467 ( #39654 )
...
cherry pick from #39467
2024-08-23 12:23:12 +08:00
021678c7c3
[fix](window_funnel) fix wrong result of window_funnel #38954 ( #39270 )
...
## Proposed changes
BP #38954
2024-08-16 09:59:31 +08:00
bf1c7a1c15
[fix](clone) fix stale tablet report miss the new cloning replica #38695 ( #38839 )
...
cherry pick from #38695
2024-08-05 18:04:24 +08:00
327069fdbc
[branch-2.1](log) add tablet clear cache log ( #38713 )
2024-08-02 08:40:02 +08:00
73199122c2
[enhancement](compaction) Control the parallelism for urgent compacton tasks ( #37782 ) ( #38189 )
...
## Proposed changes
For some urgent compaction tasks, their submittion should take
parallelism into account.
Currently, we apply the control policy for data loading in specific.
Other source of urgent tasks are considered as eager.
2024-07-22 17:22:53 +08:00
b272247a57
[pick]log thread num ( #37258 )
...
## Proposed changes
pick #37159
2024-07-04 15:27:52 +08:00
22cb7b8fcb
[improvement](compaction) be do not compact invisible version to avoid query error -230 #28082 ( #36222 )
...
cherry pick from #28082
2024-06-27 13:45:21 +08:00
630fd06ccf
Pick "[feature](Resource) Support to specify the root path for hdfs resource #32632 " ( #35848 )
...
same as #32632
2024-06-05 08:59:20 +08:00
5c40e87667
[opt](s3) auto retry when meeting 429 error ( #35397 )
...
- Add 2 new BE config
- `s3_read_base_wait_time_ms` and `s3_read_max_wait_time_ms`
When meet s3 429 error, the "get" request will
sleep `s3_read_base_wait_time_ms (*1, *2, *3, *4)` ms get try again.
The max sleep time is s3_read_max_wait_time_ms
and the max retry time is max_s3_client_retry
- Add more metrics for s3 file reader
- `s3_file_reader_too_many_request`: counter of 429 error.
- `s3_file_reader_s3_get_request`: the QPS of s3 get request.
- `TotalGetRequest`: Get request counter in profile
- `TooManyRequestErr`: 429 error counter in profile
- `TooManyRequestSleepTime`: Sum of sleep time after 429 error in profile
- `TotalBytesRead`: Total bytes read from s3 in profile
2024-05-28 23:00:31 +08:00
0d2ab9d5c3
[fix](clean trash) Fix clean trash lost submit task ( #35271 )
2024-05-23 16:27:20 +08:00
2ed6a00fd1
[opt](memory) Add GlobalMemoryArbitrator and support ReserveMemory ( #34985 ) ( #35070 )
2024-05-22 09:53:45 +08:00
6c515e0c76
[fix](group commit) Make compatibility issues on serializing and deserializing wal file more clear ( #34793 )
2024-05-18 18:12:43 +08:00
7a40f2a547
[branch-2.1](resource)fix check available fail when s3 aws_token is set and reset as, sk faild on be. ( #34219 )
2024-05-09 19:06:14 +08:00
8fdfbcb3c4
Revert "[Opt](func) opt the percentile func performance ( #34373 ) ( #34416 )"
...
This reverts commit 509ae425e416b4779ae94eab9c2b21f9850e03c3.
2024-05-07 07:23:48 +08:00
509ae425e4
[Opt](func) opt the percentile func performance ( #34373 ) ( #34416 )
2024-05-06 20:10:35 +08:00
5f2d0e3d53
[Fix](executor)Fix when Fe send empty wg list to be may cause query failed. ( #34074 )
2024-04-25 12:01:44 +08:00
a050513c91
[Fix](clean trash) Fix clean trash use agent task ( #33912 ) ( #33972 )
...
* [Fix](clean trash) Fix clean trash use agent task (#33912 )
* add .h
2024-04-22 17:14:21 +08:00
20b37e7a18
Add workload group id in workload policy's property ( #33483 )
2024-04-17 23:42:14 +08:00
58347aa2b7
[branch-2.1](bugfix) fix alter tablet not have memtracker
...
[branch-2.1](memory) Fix EngineAlterTabletTask attach memory tracker #33506
2024-04-11 09:29:42 +08:00
f3a60374b5
[branch-2.1](memory) Fix EngineStorageMigrationTask MemTracker attach thread ( #33509 )
2024-04-11 08:35:13 +08:00
cf7595d423
[opt](memory) Optimize mem tracker accuracy ( #32039 ) ( #33140 )
2024-04-10 11:42:19 +08:00
d60d804d9c
[fix](memory) Fix task repeat attach task DCHECK failed #32784 ( #33343 )
...
[branch-2.1](memory) Fix CCR task repeat attach task DCHECK failed3 #33366
2024-04-08 16:15:04 +08:00
309de6c82b
[fix](smooth-upgrade) Fix function now upgrade compatible #32807
2024-03-27 10:49:34 +08:00
e71e1b6f30
[enhancement](threadname) change thread name more simple in order to see it in top -H ( #31734 )
...
Co-authored-by: yiguolei <yiguolei@gmail.com >
2024-03-25 22:31:55 +08:00
ab467f53db
[fix](partition) Fix be tablet partition id eq 0 By report tablet ( #32179 ) ( #32667 )
2024-03-22 15:38:58 +08:00
0cde0cbf19
(invert index) modify of time series compaction policy
2024-03-22 08:16:30 +08:00
5ceccb5ba5
[fix](compatibility) should enable windown funnel mode from 2.0 ( #32284 )
2024-03-16 20:56:16 +08:00
62023d705d
[refactor](rename) rename task group to workload group in be ( #32204 )
...
---------
Co-authored-by: yiguolei <yiguolei@gmail.com >
2024-03-15 18:04:02 +08:00
847ec368be
[Fix](smooth-upgrade) Fix incompatibility when upgrade from 2.0 to 2.1 ( #32220 )
2024-03-14 11:23:05 +08:00
d8b9909675
[Fix](Status) Handle returned Status correctly #31434
2024-03-01 04:25:43 +08:00
41e31ee333
creat hdfs fs with it's resource id ( #31505 )
2024-02-28 11:33:34 +08:00
c34639245e
[Improvement](executor)add remote scan thread pool ( #31376 )
...
* add remote scan thread pool
* +1
2024-02-27 10:12:33 +08:00
1127b0065a
[Improment](executor)Add scanbytes/scanrows condition ( #31364 )
...
* Add scanbytes/scanrows condition
* fix reg
2024-02-27 10:12:33 +08:00
b8fe620ba3
Fix cgroup cpu controller file leak ( #31398 )
2024-02-27 10:12:33 +08:00
859e56ac16
[bugfix](wg) should set task group down after thread pool stopped
2024-02-25 18:09:39 +08:00
52c45e38af
[Refactor](RF) refactor the profile of rf and pipeline-x support local ignore ( #31287 )
...
* [Refactor](RF) refactor the profile of rf and pipeline-x support local ignore
* fix local merge filter
2024-02-23 19:05:06 +08:00
c56cb0ac3e
[Exec](RF) Support merge remote rf local first ( #31067 )
2024-02-22 13:01:48 +08:00
69c7750db2
[Improment](executor)Create workload thread pool without cgroup #31170
2024-02-21 13:53:18 +08:00
6cf7468073
[enhancement](function) change some function nullable mode ( #30991 )
...
change some function nullable mode
2024-02-18 14:45:25 +08:00
94eedd8ea4
[Enhancement](function)make SUBSTRING_INDEX function DEPEND_ON_ARGUMENT ( #30392 )
2024-02-02 13:31:47 +08:00
cc3c6d1479
[improvement](create tablet) backend create tablet round robin among … ( #30530 )
...
* [improvement](create tablet) backend create tablet round robin among … (#29818 )
* [improvement](create tablet) be choose disk tolerate with little skew (#30354 )
---------
Co-authored-by: yujun <yu.jun.reach@gmail.com >
2024-01-30 10:20:35 +08:00
2e6a00690f
[Fix](smooth-upgrade) fix unnecessary high version of smooth upgrade ( #30283 )
...
fix unnecessary high version of smooth upgrade
2024-01-25 13:24:09 +08:00
ca5a314765
[fix](function) make STRLEFT and STRRIGHT and SUBSTR function DEPEND_ON_ARGUMENT ( #28352 )
...
make STRLEFT and STRRIGHT function DEPEND_ON_ARGUMENT
2024-01-25 13:23:59 +08:00
72f4e7e2d1
[security] Don't print token ( #30227 )
2024-01-24 09:59:45 +08:00
9e0c518aaf
[Feature](executor)Workload Group support Non-Pipeline Execution ( #30164 )
2024-01-23 10:11:25 +08:00
8ca807578f
[fix](migrate disk) fix migrate disk lost data during publish version ( #29887 )
...
Co-authored-by: Yongqiang YANG <98214048+dataroaring@users.noreply.github.com >
2024-01-16 18:37:06 +08:00
5ef8428345
[Refactor](executor)refactor workload group log fron WARNING to INFO #29878
2024-01-12 13:58:19 +08:00
