Commit Graph

1264 Commits

Author SHA1 Message Date
e8f479882d [pipelineX](local exchange) Add local exchange operator (#25846) 2023-10-25 18:45:02 +08:00
693982fd1a [feature](decimal) support decimal256 (#25386) 2023-10-25 15:47:51 +08:00
97c2fe75d1 [feature](pipelineX) use expected<T, Status> in local_state (#25878) 2023-10-25 15:23:17 +08:00
235ae9ded4 [improvement](fragment) optimize to get query context logic (#25621) 2023-10-25 14:03:47 +08:00
5e3277e8fb [improvement](routine-load) add routine load rows check (#25818) 2023-10-25 11:04:28 +08:00
552091f21f [performance](pipelineX) optimize pipelineX (#25713) 2023-10-25 10:13:17 +08:00
6b2eed779c [feature](AuditLog) add scanRows scanBytes in auditlog (#25435) 2023-10-25 10:00:35 +08:00
69c3e08699 [improve](load) set load id in signal handler (#25813) 2023-10-24 21:58:39 +08:00
Pxl
2972daaed9 [Bug](status) process error status on es_scroll_parser and compaction_action (#25745)
process error status on es_scroll_parser and compaction_action
2023-10-24 15:51:01 +08:00
87b414cdae [Fix](query execution) Fix result sink fragment can't be cancelled in non-pipeline (#25524) 2023-10-24 11:30:29 +08:00
215c1625b2 [chore](fragmentid) rename fragment id to fragment instance id in thrift definition file (#25787)
* rename fragment id to fragment instance id in thrift definition file

* f

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
2023-10-24 11:26:52 +08:00
d0da94e22b [refactor](streamload) refactor stream load executor (#25615) 2023-10-23 14:34:26 +08:00
Pxl
642c149e6a remove datetime_value and move vecdatetime_value to doris namespace (#25695)
remove datetime_value and move vecdatetime_value to doris namespace
2023-10-20 22:08:17 +08:00
fded8ba824 [improvement](insert) Modify insert into table_id(xxx) to doris_inter… (#25662) 2023-10-20 19:31:37 +08:00
d0cd535cb9 [improvement](insert) refactor group commit stream load (#25560) 2023-10-20 13:27:30 +08:00
b0e70bd02b [fix](pipelineX) fix error set delta_urls (#25631) 2023-10-19 19:27:31 +08:00
54780c62e0 [improvement](executor)Using cgroup to implement cpu hard limit (#25489)
* Using cgroup to implement cpu hard limit

* code style
2023-10-19 18:56:26 +08:00
159be51ea6 [bugfix](schema_change) Fix the coredump when doubly write during schema change (#22557) 2023-10-19 14:43:18 +08:00
9c9fc84f39 [feature](merge-cloud) Abstract BaseTablet for CloudTablet (#24929) 2023-10-18 20:29:04 +08:00
b0e0a0569a [Fix](row store) Real default value should be used instead of default… (#25230)
Before this PR the default value is not correct, we should use default value in Frontend schema.
2023-10-18 10:13:44 +08:00
18c2a13e09 [fix](multi-catalog)fix maxcompute partition filter and session creation (#24911)
add maxcompute partition support
fix maxcompute partition filter
modify maxcompute session create method
2023-10-17 22:36:10 +08:00
f1a5e393c7 [feature](insert) Support group commit insert use new syntax like insert into table_id(xxx) (#25484) 2023-10-17 09:23:09 +08:00
59ebbb351e [feature](merge-cloud) Enable write into cache when uploading file to s3 using s3 file writer (#24364) 2023-10-16 21:31:02 +08:00
7ea456ef91 [fix](insert) make group commit wal_manager exit elegantly (#25250) 2023-10-14 23:14:06 +08:00
e5ef0aa6d4 [refactor](mysql result format) use new serde framework to tuple convert (#25006) 2023-10-14 19:46:42 +08:00
2ec53ff60e [fix](multi-table) fix single stream multi table load can not finish (#25379) 2023-10-13 15:47:16 +08:00
1073ef22f3 [fix](insert) improve group_commit related tests (#25319) 2023-10-12 21:19:29 +08:00
022762d5f0 [fix](memory) Fix work load group GC and add logs to locate slow GC #24975
Fix work load group GC, add cancel load and add logs.
Unify the format and change all to lowercase of GC logs, avoid unnecessary trouble when grep or less
Add logs to help locate the cause of slow GC.
2023-10-12 10:33:56 +08:00
73f632a4e3 [fix](move-memtable) handle error in LoadStreamWriter::close (#24805) 2023-10-11 16:54:42 +08:00
bb670118f5 [coverage](test) Delete unused function to improve test coverage (#25233) 2023-10-11 11:50:51 +08:00
880d0d7e70 [Bug](pipeline) Support the auto partition in pipeline load (#25176) 2023-10-10 11:51:12 +08:00
4de3df6a46 [refactor](column) remove unused method and column definitions (#25152)
remove unused method and column definitions
using primitive type in predicate column to check datev1 and datev2
2023-10-09 17:14:35 +08:00
9d8b993c51 [fix](fs) fix remove error log failed (#25108) 2023-10-08 22:15:37 +08:00
6fe060b79e [fix](streamload) fix http_stream retry mechanism (#24978)
If a failure occurs, doris may retry. Due to ctx->is_read_schema is a global variable that has not been reset in a timely manner, which may cause exceptions.


---------

Co-authored-by: yiguolei <676222867@qq.com>
2023-10-08 11:16:21 +08:00
8953179c11 [fix](multi-table) fix multi table task cannot end (#25056)
When exec multi table task, it can not end when exec plan error, which causes other routine load task can not submit.
2023-10-07 19:45:42 +08:00
59261174d5 [chore](unused) Remove unused variable CPU_HARD_LIMIT in task_group.cc (#25076)
Signed-off-by: Jack Drogon <jack.xsuperman@gmail.com>
2023-10-07 03:36:13 -05:00
7b2ff38401 query cpu hard limit based on doris scheduler (#24844) 2023-10-07 12:03:07 +08:00
4ce5213b1c [fix](insert) Fix test_group_commit_stream_load and add more regression in test_group_commit_http_stream (#24954) 2023-10-03 20:56:24 +08:00
642e5cdb69 [Fix](Status) Make Status [[nodiscard]] and handle returned Status correctly (#23395) 2023-09-29 22:38:52 +08:00
864a0f9bcb [opt](pipeline) Make pipeline fragment context send_report asynchronized (#23142) 2023-09-28 17:55:53 +08:00
1b0e3246ea [pipelineX](fix) Fix exception reporting and Nereids plan (#24936) 2023-09-27 13:15:40 +08:00
55d1090137 [feature](insert) Support group commit stream load (#24304) 2023-09-26 20:57:02 +08:00
082bcd820b [feature](insert) Support wal for group commit insert (#23053) 2023-09-26 14:46:24 +08:00
8d4fd76a16 [Feature](StreamLoad2PC) Support commit and abort streamload2PC by label (#24613) 2023-09-25 22:21:27 +08:00
6502da8917 [bugfix](restore) add partition id into convert_rowset_ids() (#24834) 2023-09-25 20:07:24 +08:00
c346f4d638 [Improvement](Storage) Lazy init mini_download dir (#24649) 2023-09-22 16:23:26 +08:00
c9b2f4cb92 [workload](pipeline) Add cgroup cpu controller (#24052) 2023-09-21 21:49:33 +08:00
8eb14eec7c [enhancement](baddisk) record bad disk in be_custom.conf to handle (#24639) 2023-09-21 18:31:58 +08:00
85fb46bb71 [refactor](cache) Refactor preloaded timezone global cache (#24694)
Refactor preloaded timezone global cache
2023-09-21 17:26:41 +08:00
b9997d69fa [refactor](pipelineX) return error when local_state can not find id (#24360) 2023-09-21 16:04:08 +08:00