database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
13,820 Commits 1 Branch 0 Tags
2ec50dcfc7e7ac328e378baedb0b7cf59dcbf16e
Commit Graph

5597 Commits

Author SHA1 Message Date
zhannngchen
2ec50dcfc7 [log](compaction) add more stats for compaction log (#24984) 2023-09-28 15:29:15 +08:00
zhangstar333
b6babf3af4 [pipelineX](sink) support jdbc table sink (#24970) 
* [pipelineX](sink) support jdbc table sink
 2023-09-28 14:39:32 +08:00
Gabriel
b35171b582 [pipelineX](bug) fix distinct streaming agg (#24995) 2023-09-28 14:01:26 +08:00
Gabriel
f0fad61db4 [pipelineX](bug) Fix file scan operator (#24989) 2023-09-28 11:12:27 +08:00
Siyang Tang
188d9ab94e [enhancement](statistics) collect table level loaded rows on BE to make RPC light weight (#24609) 2023-09-28 10:51:50 +08:00
Mryange
430634367a [pipelineX](node)support file scan operator (#24924) 2023-09-27 22:10:43 +08:00
Chenyang Sun
68087f6c82 [fix](json function) Fix the slow performance of get_json_path when processing JSONB (#24631) 
When processing JSONB, automatically convert to jsonb_extract_string
 2023-09-27 21:17:39 +08:00
Liqf
d4e823950a [bug](json)Fix some problems of json function on Nereids (#24898) 
Fix some problems of json_length and json_contains function on Nereids
fix wrong result of json_contains function
Regression test jsonb_p0 to enable Nereids
 2023-09-27 21:01:45 +08:00
Gabriel
947b116318 [pipelineX](fix) Fix BE crash due to ES scan operator (#24983) 2023-09-27 20:45:38 +08:00
Gabriel
1fb9022d07 [pipelineX](bug) Fix meta scan operator (#24963) 2023-09-27 20:34:47 +08:00
Gabriel
671b5f0a0a [Bug](pipeline) Fix block reusing for union source operator (#24977) 
[CANCELLED][INTERNAL_ERROR]Merge block not match, self:[String], input:[String, Nullable(String), Nullable(String), Nullable(String), Nullable(String), DateV2]
 2023-09-27 19:41:56 +08:00
Mryange
5d138b6928 [remove](function) make execute_impl const and remove running_difference function (#24935) 2023-09-27 18:17:28 +08:00
huanghaibin
c04078f3b8 [improvement](compaction) output tablet_id when be core dumped. (#24952) 2023-09-27 16:50:18 +08:00
gohalo
19cff5d167 [fix](compile) failed on arm platform, with clang compiler and pch on (#24636) 
failed on arm platform, with clang compiler and pch on
 2023-09-27 16:47:02 +08:00
Pxl
5fc04b6aeb [Improvement](hash) some refactor of process hash table probe impl (#24461) 
some refactor of process hash table probe impl
 2023-09-27 16:14:49 +08:00
Gabriel
aa4dbbedc7 [pipelineX](bug) Fix dead lock in exchange sink operator (#24947) 2023-09-27 15:40:25 +08:00
Xinyi Zou
87a30dc41d [feature-wip](arrow-flight)(step3) Support authentication and user session (#24772) 2023-09-27 14:53:58 +08:00
Ashin Gau
26818de9c8 [feature](jni) support complex types in jni framework (#24810) 
Support complex types in jni framework, and successfully run end-to-end on hudi.
### How to Use
Other scanners only need to implement three interfaces in `ColumnValue`:
```
// Get array elements and append into values
void unpackArray(List<ColumnValue> values);

// Get map key array&value array, and append into keys&values
void unpackMap(List<ColumnValue> keys, List<ColumnValue> values);

// Get the struct fields specified by `structFieldIndex`, and append into values
void unpackStruct(List<Integer> structFieldIndex, List<ColumnValue> values);
```
Developers can take `HudiColumnValue` as an example.
 2023-09-27 14:47:41 +08:00
Gabriel
1b0e3246ea [pipelineX](fix) Fix exception reporting and Nereids plan (#24936) 2023-09-27 13:15:40 +08:00
zhangstar333
c04e5bac39 [bug](pipelineX) fix java-udaf failed with open pipelineX (#24939) 2023-09-27 13:14:10 +08:00
zzzzzzzs
452318a9fc [Enhancement](streamload) stream tvf support user specified label (#24219) 
stream tvf support user specified label
example:

curl -v --location-trusted -u root: -H "sql: insert into test.t1 WITH LABEL label1 select c1,c2 from http_stream(\"format\" = \"CSV\", \"column_separator\" = \",\")" -T example.csv http://127.0.0.1:8030/api/_http_stream
return:

{
    "TxnId": 2064,
    "Label": "label1",
    "Comment": "",
    "TwoPhaseCommit": "false",
    "Status": "Success",
    "Message": "OK",
    "NumberTotalRows": 2,
    "NumberLoadedRows": 2,
    "NumberFilteredRows": 0,
    "NumberUnselectedRows": 0,
    "LoadBytes": 27,
    "LoadTimeMs": 152,
    "BeginTxnTimeMs": 0,
    "StreamLoadPutTimeMs": 83,
    "ReadDataTimeMs": 92,
    "WriteDataTimeMs": 41,
    "CommitAndPublishTimeMs": 24
}
 2023-09-27 12:09:35 +08:00
HappenLee
24ee3607e1 [Bug](pipeline) nullprt may be close the sink if init failed (#24926) 2023-09-27 09:11:06 +08:00
Gabriel
a689a2fbb1 [pipelineX](fix) Fix projection expression (#24923) 2023-09-26 21:48:28 +08:00
meiyi
55d1090137 [feature](insert) Support group commit stream load (#24304) 2023-09-26 20:57:02 +08:00
zhannngchen
fe2879d8fe [fix](merge-on-write) MergeIndexDeleteBitmapCalculator stack overflow (#24913) 2023-09-26 20:32:23 +08:00
Siyang Tang
77e864df12 [enhancement](delete) use column id in delete push task instead of column name (#24549) 2023-09-26 19:54:55 +08:00
Tiewei Fang
28869b0f82 [fix](Outfile) Use data_type_serde to export data to orc file format (#24812) 2023-09-26 19:46:42 +08:00
airborne12
94082ae59c [Fix](inverted index) fix tokenize function coredump (#24896) 2023-09-26 17:31:10 +08:00
Gabriel
156b7b80a4 [pipelineX](bug) Fix scan operator if aggregation type is COUNT_ON_INDEX (#24909) 2023-09-26 16:42:29 +08:00
bobhan1
1abda1c446 [Fix](merge-on-write) Correct the alignment process when the existing rows with same key has marked delete sign (#24877) 2023-09-26 16:09:20 +08:00
huanghaibin
082bcd820b [feature](insert) Support wal for group commit insert (#23053) 2023-09-26 14:46:24 +08:00
AlexYue
422170ccb1 [Bugfix](tablet) change WriteCooldownMetaExecutors's thread pool type to prevent singleton's dtor order (#23834) 2023-09-26 14:29:35 +08:00
Gabriel
a3427cb822 [pipelineX](fix) Fix nested loop join operator (#24885) 2023-09-26 13:27:34 +08:00
Mryange
733b71828c [fix](pipelineX) fix do not set per_fragment_instance_idx (#24890) 2023-09-26 13:10:30 +08:00
zhangstar333
513e37bdbf [pipelineX](node)support jdbc scan operator (#24851) 2023-09-26 10:02:51 +08:00
Yongqiang YANG
8ee2e40812 [enhancement](zone_map_index) use direct_copy_may_cut instead of direct_copy(#24880) 2023-09-25 22:32:18 +08:00
plat1ko
8191cd1dad [Bug](ScanNode) Fix potential incorrect query result caused by concurrent NewOlapScanNode initialization and Compaction (#24638) 
* Optimize fetch delete predicates

* Fix incorrect query result when compaction eliminate delete predicates between `NewOlapScanNode::_init_scanners` and `NewOlapScanner::init`

* Fix be ut
 2023-09-25 22:24:35 +08:00
HHoflittlefish777
8d4fd76a16 [Feature](StreamLoad2PC) Support commit and abort streamload2PC by label (#24613) 2023-09-25 22:21:27 +08:00
Gabriel
b38b8b4494 [pipelineX](fix) Fix BE crash caused by join and constant expr (#24862) 2023-09-25 21:01:09 +08:00
DeadlineFen
6502da8917 [bugfix](restore) add partition id into convert_rowset_ids() (#24834) 2023-09-25 20:07:24 +08:00
yujun
8679095e5c [feature](debug) support debug point used in debug code (#24502) 2023-09-25 17:56:12 +08:00
Gabriel
3b4d8b4ac8 [pipelineX](feature) Support schema scan operator (#24850) 2023-09-25 14:42:25 +08:00
wangbo
9412775686 remove useless variable in scanctx (#24849) 
remove useless variable in scanctx
 2023-09-25 14:36:18 +08:00
xfz
1b95ce1d93 [feature](json-function) add json_insert, json_replace, json_set functions (#24384) 
[feature](json-function) add three json funcitons
 2023-09-25 12:52:29 +08:00
zhangstar333
c9ef5ef2b1 [refactor](profile) refactor join node profile when build side shared hash table (#24785) 
refactor join node profile when build side shared hash table
 2023-09-25 10:28:16 +08:00
xy720
39e6512a21 [bug](scanner) Fix memory out of bound in scanner scheduler (#24840) 2023-09-25 09:58:26 +08:00
HappenLee
9579634eac [Debug](pipeline) add log of pipeline scan bug (#24804) 2023-09-25 08:38:31 +08:00
yujun
900b06d14e [improvement](disk balancer) reduce disk balance sensitivity (#24637) 2023-09-24 22:02:53 +08:00
bobhan1
fa0e0a606b [Fix](autoinc) Ensure that _fetch_autoinc_id_executor is destructed after the destructions of AutoIncIDBuffers (#24832) 2023-09-24 21:44:02 +08:00
Gabriel
27eed937b3 [pipelineX](es scan) Support ES scan operator (#24824) 
Support ES scan operator
 2023-09-24 00:32:38 +08:00