database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
16,239 Commits 1 Branch 0 Tags
d50c8b6d3ac39780e8bd0a235da2ec79cea93308
Commit Graph

340 Commits

Author SHA1 Message Date
AlexYue
d61fec3ff8 (feature)(LOG) Support to change be vlog level dynamically using http (#28785) 2024-01-06 20:12:35 +08:00
huanghaibin
3888a7cc0b [fix](group_commit) Fix check auth error when relaying wal (#29461) 2024-01-03 23:19:16 +08:00
abmdocrt
e3c9f535dc [refactor](wal) refactor some wal code (#29434) 2024-01-03 14:45:57 +08:00
Xinyi Zou
be1d9c3358 [fix](memory) Fix mem tracker web page notice #29361 2024-01-03 11:25:00 +08:00
abmdocrt
b3f69211b7 [refactor](wal) move group commit load content length to runtime state (#29188) 2024-01-02 15:53:38 +08:00
meiyi
706463781c [refactor](group commit) refactor group commit wal code (#29375) 2024-01-02 15:52:03 +08:00
Nitin-Kashyap
018d8b1326 [fix](doc) spell errors fixes and align with code log for check-rpc-channel. (#28151) 2023-12-30 20:56:56 +08:00
huanghaibin
03901b9a7a [enhancement](group_commit): refector relay wal code (#29183) 2023-12-30 12:59:46 +08:00
py023
e7d67e9411 [fix](be) resolves some unused-raii and used-after-moved issues (#29285) 2023-12-30 12:14:49 +08:00
meiyi
9be0f04506 (improv)[group commit] refactor some group commit code (#29180) 2023-12-29 00:26:10 +08:00
abmdocrt
9ff8bd2e9c [Enhancement](Wal)Support dynamic wal space limit (#27726) 2023-12-27 11:51:32 +08:00
zclllyybb
f374beaa4e [fix](log) regularise some BE error type and fix a load task check #28729 2023-12-25 10:45:19 +08:00
xy
e59c707022 [fix](stream_load)fix bug for stream (#27752) 
1. forbid thed  stream_load without content-length or chunked Transfer Encoding
2. forbid thed  stream_load both with content-length and chunked Transfer Encoding

Co-authored-by: xingying01 <xingying01@corp.netease.com>
 2023-12-20 09:47:52 +08:00
yujun
f9ddf8c7ef [improvement](be report) add be report http (#28424) 2023-12-19 10:39:19 +08:00
meiyi
ee24667b9f [fix](group commit) Fix some group commit problems (#28319) 2023-12-14 14:38:56 +08:00
meiyi
d4f89389e3 [improve](group commit) Group commit support skip wal (#27957) 2023-12-11 19:38:32 +08:00
abmdocrt
605257ccb7 [Enhancement](group commit) Add regression case for wal limit (#27949) 2023-12-06 14:23:50 +08:00
plat1ko
1afdbfe723 [enhance](BE) Refactor TaskWorkerPool (#27555) 2023-12-04 21:46:10 +08:00
abmdocrt
a64656748b [Enhancenment](wal) disable group commit when streamload size is too large (#27781) 2023-12-03 23:05:11 +08:00
Mingyu Chen
9daa7dc6b5 [refactor](http) disable snapshot and get_log_file api (#27724) 
Disable 2 http api by default:

1. BE's `/api/snapshot`
2. FE's `/get_log_file`
 2023-11-29 16:11:51 +08:00
ShowCode
f565f60bc3 [refactor](standard)BE:Initialize pointer variables in the class to nullptr by default (#27587) 2023-11-28 13:02:30 +08:00
Gabriel
840f3b6439 [pipelineX](dependency) Wake by task by read dependency (#27260) 2023-11-20 22:00:59 +08:00
Gabriel
b1eef30b49 [pipelineX](dependency) Wake up task by dependencies (#26879) 
---------

Co-authored-by: Mryange <2319153948@qq.com>
 2023-11-18 03:20:24 +08:00
camby
e1b180d53d [improve](streamload) Explicitly judge the return value of close #27134 2023-11-17 14:17:09 +08:00
Guangdong Liu
e1ba471727 [fix](send_batch_parallelism) add test case for send_batch_parallelism (#26908) 2023-11-15 14:21:58 +08:00
meiyi
c0fda8c5c2 [improve](group commit) Add a swicth to wait internal group commit lo… (#26734) 
* [improve](group commit) Add a swicth to make internal group commit load finish

* modify group commit tvf plan
 2023-11-13 10:35:35 +08:00
plat1ko
d767804815 [feature](merge-cloud) Decouple rowset id generator and local rowsets gc implementation (#25921) 2023-11-10 10:07:02 +08:00
HHoflittlefish777
607a5d25f1 [feature](streamload) support HTTP request with chunked transfer (#26520) 2023-11-08 10:07:05 +08:00
Jack Drogon
2cc68381ec [feature](binlog) Add ingest_binlog/http_get_snapshot limit download speed && Add async ingest_binlog (#26323) 2023-11-06 11:14:44 +08:00
Pxl
696ecc8c83 [Chore](log) adjust error code on too many filtered rows (#26168) 2023-11-01 00:15:56 +08:00
Siyang Tang
88dd480c2e [enhancement](CSV-reader) enhance err log for csv reading containing enclose or escape (#25816) 2023-10-24 22:10:08 +08:00
Pxl
2972daaed9 [Bug](status) process error status on es_scroll_parser and compaction_action (#25745) 
process error status on es_scroll_parser and compaction_action
 2023-10-24 15:51:01 +08:00
Pxl
2e2d5bcba2 [Improvements](status) catch some error status (#25677) 
catch some error status
 2023-10-23 10:19:08 +08:00
meiyi
d0cd535cb9 [improvement](insert) refactor group commit stream load (#25560) 2023-10-20 13:27:30 +08:00
plat1ko
9c9fc84f39 [feature](merge-cloud) Abstract BaseTablet for CloudTablet (#24929) 2023-10-18 20:29:04 +08:00
Kaijie Chen
cda8fb6b8b [fix](load) return Status when error in RowsetWriter::build (#25381) 2023-10-17 09:40:23 +08:00
yujun
73c3e3ab55 [Feature](x-load) support config min replica num for loading data (#21118) 2023-10-11 21:07:35 +08:00
zzzzzzzs
5a55e47acd [Enhancement](Load) stream tvf support two phase commit (#23800) 2023-10-09 14:15:56 +08:00
zzzzzzzs
6fe060b79e [fix](streamload) fix http_stream retry mechanism (#24978) 
If a failure occurs, doris may retry. Due to ctx->is_read_schema is a global variable that has not been reset in a timely manner, which may cause exceptions.


---------

Co-authored-by: yiguolei <676222867@qq.com>
 2023-10-08 11:16:21 +08:00
bobhan1
642e5cdb69 [Fix](Status) Make Status [[nodiscard]] and handle returned Status correctly (#23395) 2023-09-29 22:38:52 +08:00
zzzzzzzs
452318a9fc [Enhancement](streamload) stream tvf support user specified label (#24219) 
stream tvf support user specified label
example:

curl -v --location-trusted -u root: -H "sql: insert into test.t1 WITH LABEL label1 select c1,c2 from http_stream(\"format\" = \"CSV\", \"column_separator\" = \",\")" -T example.csv http://127.0.0.1:8030/api/_http_stream
return:

{
    "TxnId": 2064,
    "Label": "label1",
    "Comment": "",
    "TwoPhaseCommit": "false",
    "Status": "Success",
    "Message": "OK",
    "NumberTotalRows": 2,
    "NumberLoadedRows": 2,
    "NumberFilteredRows": 0,
    "NumberUnselectedRows": 0,
    "LoadBytes": 27,
    "LoadTimeMs": 152,
    "BeginTxnTimeMs": 0,
    "StreamLoadPutTimeMs": 83,
    "ReadDataTimeMs": 92,
    "WriteDataTimeMs": 41,
    "CommitAndPublishTimeMs": 24
}
 2023-09-27 12:09:35 +08:00
meiyi
55d1090137 [feature](insert) Support group commit stream load (#24304) 2023-09-26 20:57:02 +08:00
huanghaibin
082bcd820b [feature](insert) Support wal for group commit insert (#23053) 2023-09-26 14:46:24 +08:00
HHoflittlefish777
8d4fd76a16 [Feature](StreamLoad2PC) Support commit and abort streamload2PC by label (#24613) 2023-09-25 22:21:27 +08:00
yujun
8679095e5c [feature](debug) support debug point used in debug code (#24502) 2023-09-25 17:56:12 +08:00
bobhan1
58ab25ccaa Revert "[Feature](merge-on-write)Support ignore mode for merge-on-write unique table (#21773)" (#24731) 
This reverts commit 3ee89aea35726197cb7e94bb4f2c36bc9d50da84.
 2023-09-21 21:01:28 +08:00
yiguolei
9c681692bd Revert "[fix] fix http_stream retry mechanism (#23969)" (#24407) 
This reverts commit 05e365ea137eb8c92b8e7eedc7d1435e83f065ae.
 2023-09-15 10:07:53 +08:00
zzzzzzzs
05e365ea13 [fix] fix http_stream retry mechanism (#23969) 
Co-authored-by: yiguolei <676222867@qq.com>
 2023-09-14 21:41:11 +08:00
zclllyybb
927de33166 [config](log) disable StreamLoad log default and enable in regression pipeline (#24354) 
disable StreamLoad log default and enable in regression pipeline
 2023-09-14 20:47:26 +08:00
bobhan1
3ee89aea35 [Feature](merge-on-write)Support ignore mode for merge-on-write unique table (#21773) 2023-09-14 18:03:51 +08:00