database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
21,044 Commits 1 Branch 0 Tags
3006eea5cc4ebec0b30b75f3ee9f52b027f1518c
Commit Graph

8449 Commits

Author SHA1 Message Date
Pxl
d70c17bdc0 [Improvement](scan) use loop to instead recursion on Level1Iterator::_normal_next #38005 (#45767) 
pick from  #38005
 2024-12-23 13:54:12 +08:00
Mryange
367ecc3292 [fix](expr)Remove the _can_fast_execute flag from VExpr. (#45542) (#45662) 2024-12-22 21:26:32 +08:00
lihangyu
39c69c766e [Optimize](Variant) optimize schema update performance (#45480) (#45731) 
(#45480)
 2024-12-21 23:41:03 +08:00
lihangyu
06efd5b4af [Opt](SegmentIterator) clear and release iterators memory footprint in advance when EOF (#44768) (#45734) 
(#44768)
 2024-12-20 20:38:56 +08:00
github-actions[bot]
37c4de3cbf branch-2.1: [improve](variant) only sanitize in debug mode #45689 (#45698) 
Cherry-picked from #45689

Co-authored-by: lihangyu <lihangyu@selectdb.com>
 2024-12-20 14:25:49 +08:00
Mingyu Chen (Rayner)
19c0e89da7 [enchement](iceberg)support read iceberg partition evolution table. (#45367) (#45569) 
cherry-pick #45367

Co-authored-by: daidai <changyuwei@selectdb.com>
 2024-12-20 08:56:51 +08:00
walter
c459ad7382 [fix](binlog) Avoid clear binlog dir #45581 (#45620) 
cherry pick from #45581
 2024-12-19 23:42:54 +08:00
HappenLee
9272c650b4 [Refactor](query) refactor lock in fragment mgr and change std::unorder_map to phmap (#45069) 
### What problem does this PR solve?

Related PR: #44821
 2024-12-19 22:27:33 +08:00
Xujian Duan
4b7c2eaa7d [branch-2.1](fix) fix incorrect result of hash join with const column (#45630) 2024-12-19 19:14:38 +08:00
github-actions[bot]
d6c629d293 branch-2.1: [fix](tvf) Tvf supports to parse the enclose character in csv files #45407 (#45570) 
Cherry-picked from #45407

Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
 2024-12-19 16:23:44 +08:00
Socrates
7d32e4f71f branch-2.1: [Fix](ORC) Not push down fixed char type in orc reader #45484 (#45525) 
cherry-pick #45484
 2024-12-19 14:06:00 +08:00
github-actions[bot]
eb67db3d25 branch-2.1: [feat](docker)Add a BE ENV item 'SKIP_CHECK_ULIMIT' for Docker to start quickly #45267 (#45468) 
Cherry-picked from #45267

Co-authored-by: FreeOnePlus <54164178+FreeOnePlus@users.noreply.github.com>
 2024-12-19 09:31:41 +08:00
Pxl
fe0cc289de [Bug](function) fix wrong result on group_concat with distinct+order_… (#45513) 2024-12-18 22:49:18 +08:00
github-actions[bot]
59b3760fdd branch-2.1: [opt](join) Check the property of nullable from intermediate row #45017 (#45476) 
Cherry-picked from #45017

Co-authored-by: Jerry Hu <hushenggang@selectdb.com>
 2024-12-18 22:40:17 +08:00
Xujian Duan
02feb16530 branch-2.1: [bug](s3) fix S3 file system gets absolute path #44965 (#45529) 
Cherry-picked from https://github.com/apache/doris/pull/44965
 2024-12-18 22:29:24 +08:00
Mingyu Chen (Rayner)
855e9a508c [fix](catalog) opt the count pushdown rule for iceberg/paimon/hive scan node (#44038) (#45564) 
bp #44038
 2024-12-18 09:54:56 +08:00
github-actions[bot]
01684ce3b1 branch-2.1: [fix](mysql-buffer) fix special buffer size with nested type #45126 (#45458) 
Cherry-picked from #45126

Co-authored-by: amory <wangqiannan@selectdb.com>
 2024-12-17 17:48:10 +08:00
Pxl
900086667f [Chore](pipeline) catch exception on task::prepare to avoid exception make backend coredump #45479 (#45516) 
…dump

### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
 2024-12-17 17:22:05 +08:00
Pxl
7856662ecf [Bug](pipeline) make sink operator process eos signals after wake_up_early #45207 (#45400) 
make sink operator process eos signals after wake_up_early #45207  (#45400)
 2024-12-17 17:21:42 +08:00
github-actions[bot]
191ef9b8b0 branch-2.1: [fix](schema-change) fix array/map/struct in schema-change not-null to null will make core #45305 (#45482) 
Cherry-picked from #45305

Co-authored-by: amory <wangqiannan@selectdb.com>
 2024-12-17 17:05:17 +08:00
Kaijie Chen
8dc845671f [test](load) injection cases should check Exception is thrown (#44713) (#45321) 
backport #44713
 2024-12-17 14:37:27 +08:00
github-actions[bot]
8faf010c64 branch-2.1: [fix](function) The parameters after the first of the mask function need to be restricted to constants #45348 (#45498) 
Cherry-picked from #45348

Co-authored-by: Jerry Hu <hushenggang@selectdb.com>
 2024-12-17 14:25:05 +08:00
zclllyybb
79662fcc94 [branch-2.1](functions) clean some ip functions code and make IS_IP_ADDRESS_IN_RANGE DEPENDS_ON_ARGUMENT (#45358) 
pick https://github.com/apache/doris/pull/35239


add special logic to deal smooth upgrade

The origin PR is https://github.com/apache/doris/pull/35239. for
branch-3.0 it was merged in 3.0.0 but forgot to register old version.
now in branch-3.0 we fix it in
https://github.com/apache/doris/pull/45428 which must be merged in
3.0.4. and do same thing in this PR which must be merged in 2.1.8.
then:
```
FROM    TO    result
217-    218+    ✅
217-    303-    💥
218+    303-    ✅
218+    304+    ✅
303-    304+    ✅
```
this is our best result.
 2024-12-17 11:51:07 +08:00
zhangstar333
c561bdb906 [cherry-pick](branch-21) fix first/last value return error with ignore null (#44996) (#45486) 
cherry-pick from master https://github.com/apache/doris/pull/44996
 2024-12-17 10:04:02 +08:00
github-actions[bot]
23c5d52b04 branch-2.1: [fix](s3) improve error msg #45360 (#45432) 
Cherry-picked from #45360

Co-authored-by: Socrates <suyiteng@selectdb.com>
 2024-12-16 14:59:08 +08:00
github-actions[bot]
1db78d4496 branch-2.1: [fix](hive) fix block decompressor bug #45289 (#45379) 
Cherry-picked from #45289

Co-authored-by: Socrates <suyiteng@selectdb.com>
 2024-12-14 19:20:55 -08:00
meiyi
6dabdd6a8c branch-2.1: [fix](txn insert) txn insert show error url #45254 (#45381) 
Cherry-picked from https://github.com/apache/doris/pull/45254
 2024-12-13 17:36:02 +08:00
wangbo
384a1a4bd7 [branch-2.1]log more detail when query hits policy (#44685) (#45366) 
pick #44685
 2024-12-13 10:24:53 +08:00
wuwenchi
667f5e6e6a [feat](iceberg)Supports using rest type catalog to read tables in unity catalog for 2.1 (#43525) (#45217) 
bp: #43525
 2024-12-12 00:49:36 -08:00
bobhan1
0c55edd3bf branch-2.1: [opt](compaction) Don't check missed rows in cumu compaction if input rowsets are not in tablet (#45279) (#45304) 
pick https://github.com/apache/doris/pull/45279
 2024-12-12 11:14:30 +08:00
github-actions[bot]
da885691ca branch-2.1: [fix](column_array)fix filter_generic in column_array func #41990 (#44983) 
Cherry-picked from #41990

Co-authored-by: amory <wangqiannan@selectdb.com>
 2024-12-12 09:52:26 +08:00
github-actions[bot]
9a4f63f180 branch-2.1: [fix](Outfile) fix the unclear error code #45119 (#45277) 
Cherry-picked from #45119

Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
 2024-12-12 09:32:26 +08:00
shee
fb407f2e94 [opt](lambda) let lambda expression support refer outer slot (#45186) 2024-12-11 18:55:49 +08:00
github-actions[bot]
44fa86040f branch-2.1: [Enhancement](compaction)Optimize compaction task permit allocation #45197 (#45243) 
Cherry-picked from #45197

Co-authored-by: abmdocrt <lianyukang@selectdb.com>
 2024-12-11 14:35:07 +08:00
abmdocrt
d0737c45f8 [cherry-pick](branch-2.1) Pick "[Fix](Rowset Id) Use a randomly generated rowset ID to handle memory write failures (#42949)" (#44086) 2024-12-10 15:48:02 +08:00
Mingyu Chen (Rayner)
e29d125200 [fix](planner) query should be cancelled if limit reached (#44338) (#45222) 
cherry-pick #44338
 2024-12-09 22:22:24 -08:00
Mingyu Chen (Rayner)
5d3f0a267a [opt](scan) unify the local and remote scan bytes stats for all scanners for 2.1 (#45167) 
pick part of #40493

TODO: not working with s3 reader
 2024-12-10 14:19:19 +08:00
wangbo
0bcb99710b [branch-2.1]Add customStdAllocator for vector/map (#41193) (#45124) 2024-12-09 18:33:51 +08:00
wangbo
977da5ca28 [branch-2.1]count tablet meta's static memory load from disk (#41429) (#45128) 
pick #41429
 2024-12-09 14:44:20 +08:00
Xinyi Zou
17bcc208e9 branch-2.1: [fix](arrow-flight-sql) Revert arrow_flight_sql group from regression test #44992 (#45089) 2024-12-08 11:37:02 +08:00
github-actions[bot]
cf2e2113db branch-2.1: [fix](move-memtable) tolerate non-open streams in close wait #44680 (#45154) 
Cherry-picked from #44680

Co-authored-by: Kaijie Chen <chenkaijie@selectdb.com>
 2024-12-08 11:35:30 +08:00
daidai
b332217584 [enchement](mc)mc catalog append netowrk config (#44194) (#45149) 
bp #44194
 2024-12-07 09:52:19 -08:00
924060929
2543af8d3b [feature](function) support orthogonal_bitmap_expr_calculate & orthogonal_bitmap_expr_calculate_count for nereids (#44991) (#45012) 
support orthogonal_bitmap_expr_calculate & orthogonal_bitmap_expr_calculate_count for nereids
 2024-12-07 15:27:48 +08:00
github-actions[bot]
8f0276809a branch-2.1: [fix](compaction) tablet is not removed in compaction queue if dropped #45085 (#45136) 
Cherry-picked from #45085

Co-authored-by: meiyi <meiyi@selectdb.com>
 2024-12-07 12:15:20 +08:00
Mingyu Chen (Rayner)
d4a6fd1850 Revert #43255 & #44615 (#45096) 
Revert "branch-2.1: [enhance](orc) Optimize ORC Predicate Pushdown for
OR-connected Predicate #43255 (#44438)"
Revert "[fix](orc) check all the cases before build_search_argument
(#44615) (#44801)"
 2024-12-06 21:14:13 +08:00
bobhan1
ebb21ef031 branch-2.1: [Fix](merge-on-write) Add defensive check before partial update #44687 (#45086) 
pick https://github.com/apache/doris/pull/44687
 2024-12-06 17:16:42 +08:00
github-actions[bot]
08c8a416ea branch-2.1: [fix](runtime_profile) fix race condition in to_thrift #45047 (#45099) 
Cherry-picked from #45047

Co-authored-by: Kaijie Chen <chenkaijie@selectdb.com>
 2024-12-06 16:25:33 +08:00
Socrates
53291bd040 branch-2.1: [fix](hudi) upgrade hudi to 0.15.0 (#44267)(#44995) (#45041) 
cherry-pick pr: #44267 #44995
 2024-12-05 23:18:37 -08:00
github-actions[bot]
f0324e2a56 branch-2.1: [improvement](information_schema)Support show default value in information_schema. #44849 (#45080) 
Cherry-picked from #44849

Co-authored-by: James <lijibing@selectdb.com>
 2024-12-06 14:54:09 +08:00
Mryange
21e21f5e3b [opt](exec) Use PASSTHROUGH to improve the concurrency of the ADAPTIV… (#44971) 
…E_PASSTHROUGH SINK. (#44925)
https://github.com/apache/doris/pull/44925
before
```
op -> local sink(1) -> local source (n)
```
now
```
op -> local passthrough(1) -> local passthrough(n) ->  local sink(n) -> local source (n)
```

profile
```
                  Pipeline  :  1(instance_num=3):
                      AGGREGATION_SINK_OPERATOR  (id=4  ,  nereids_id=255):
                          CROSS_JOIN_OPERATOR  (id=3  ,  nereids_id=245):
                              LOCAL_EXCHANGE_OPERATOR  (ADAPTIVE_PASSTHROUGH)  (id=-5):
                  Pipeline  :  2(instance_num=3):
                      LOCAL_EXCHANGE_SINK_OPERATOR  (ADAPTIVE_PASSTHROUGH)  (id=-5):
                          LOCAL_EXCHANGE_OPERATOR  (PASSTHROUGH)  (id=-6):
                  Pipeline  :  3(instance_num=1):
                      LOCAL_EXCHANGE_SINK_OPERATOR  (PASSTHROUGH)  (id=-6):
                          OLAP_SCAN_OPERATOR  (id=2.  nereids_id=234.  table  name  =  nums1(nums1)):
```
 2024-12-06 09:57:31 +08:00