database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
21,026 Commits 1 Branch 0 Tags
f733aa5dcf7670bf705c83fb48eb2b1e650f8a13
Commit Graph

8618 Commits

Author SHA1 Message Date
github-actions[bot]
37c4de3cbf branch-2.1: [improve](variant) only sanitize in debug mode #45689 (#45698) 
Cherry-picked from #45689

Co-authored-by: lihangyu <lihangyu@selectdb.com>
 2024-12-20 14:25:49 +08:00
Mingyu Chen (Rayner)
19c0e89da7 [enchement](iceberg)support read iceberg partition evolution table. (#45367) (#45569) 
cherry-pick #45367

Co-authored-by: daidai <changyuwei@selectdb.com>
 2024-12-20 08:56:51 +08:00
walter
c459ad7382 [fix](binlog) Avoid clear binlog dir #45581 (#45620) 
cherry pick from #45581
 2024-12-19 23:42:54 +08:00
HappenLee
9272c650b4 [Refactor](query) refactor lock in fragment mgr and change std::unorder_map to phmap (#45069) 
### What problem does this PR solve?

Related PR: #44821
 2024-12-19 22:27:33 +08:00
Xujian Duan
4b7c2eaa7d [branch-2.1](fix) fix incorrect result of hash join with const column (#45630) 2024-12-19 19:14:38 +08:00
github-actions[bot]
d6c629d293 branch-2.1: [fix](tvf) Tvf supports to parse the enclose character in csv files #45407 (#45570) 
Cherry-picked from #45407

Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
 2024-12-19 16:23:44 +08:00
Socrates
7d32e4f71f branch-2.1: [Fix](ORC) Not push down fixed char type in orc reader #45484 (#45525) 
cherry-pick #45484
 2024-12-19 14:06:00 +08:00
github-actions[bot]
eb67db3d25 branch-2.1: [feat](docker)Add a BE ENV item 'SKIP_CHECK_ULIMIT' for Docker to start quickly #45267 (#45468) 
Cherry-picked from #45267

Co-authored-by: FreeOnePlus <54164178+FreeOnePlus@users.noreply.github.com>
 2024-12-19 09:31:41 +08:00
Pxl
fe0cc289de [Bug](function) fix wrong result on group_concat with distinct+order_… (#45513) 2024-12-18 22:49:18 +08:00
github-actions[bot]
59b3760fdd branch-2.1: [opt](join) Check the property of nullable from intermediate row #45017 (#45476) 
Cherry-picked from #45017

Co-authored-by: Jerry Hu <hushenggang@selectdb.com>
 2024-12-18 22:40:17 +08:00
Xujian Duan
02feb16530 branch-2.1: [bug](s3) fix S3 file system gets absolute path #44965 (#45529) 
Cherry-picked from https://github.com/apache/doris/pull/44965
 2024-12-18 22:29:24 +08:00
Mingyu Chen (Rayner)
855e9a508c [fix](catalog) opt the count pushdown rule for iceberg/paimon/hive scan node (#44038) (#45564) 
bp #44038
 2024-12-18 09:54:56 +08:00
github-actions[bot]
01684ce3b1 branch-2.1: [fix](mysql-buffer) fix special buffer size with nested type #45126 (#45458) 
Cherry-picked from #45126

Co-authored-by: amory <wangqiannan@selectdb.com>
 2024-12-17 17:48:10 +08:00
Pxl
900086667f [Chore](pipeline) catch exception on task::prepare to avoid exception make backend coredump #45479 (#45516) 
…dump

### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
 2024-12-17 17:22:05 +08:00
Pxl
7856662ecf [Bug](pipeline) make sink operator process eos signals after wake_up_early #45207 (#45400) 
make sink operator process eos signals after wake_up_early #45207  (#45400)
 2024-12-17 17:21:42 +08:00
github-actions[bot]
191ef9b8b0 branch-2.1: [fix](schema-change) fix array/map/struct in schema-change not-null to null will make core #45305 (#45482) 
Cherry-picked from #45305

Co-authored-by: amory <wangqiannan@selectdb.com>
 2024-12-17 17:05:17 +08:00
Kaijie Chen
8dc845671f [test](load) injection cases should check Exception is thrown (#44713) (#45321) 
backport #44713
 2024-12-17 14:37:27 +08:00
github-actions[bot]
8faf010c64 branch-2.1: [fix](function) The parameters after the first of the mask function need to be restricted to constants #45348 (#45498) 
Cherry-picked from #45348

Co-authored-by: Jerry Hu <hushenggang@selectdb.com>
 2024-12-17 14:25:05 +08:00
zclllyybb
79662fcc94 [branch-2.1](functions) clean some ip functions code and make IS_IP_ADDRESS_IN_RANGE DEPENDS_ON_ARGUMENT (#45358) 
pick https://github.com/apache/doris/pull/35239


add special logic to deal smooth upgrade

The origin PR is https://github.com/apache/doris/pull/35239. for
branch-3.0 it was merged in 3.0.0 but forgot to register old version.
now in branch-3.0 we fix it in
https://github.com/apache/doris/pull/45428 which must be merged in
3.0.4. and do same thing in this PR which must be merged in 2.1.8.
then:
```
FROM    TO    result
217-    218+    ✅
217-    303-    💥
218+    303-    ✅
218+    304+    ✅
303-    304+    ✅
```
this is our best result.
 2024-12-17 11:51:07 +08:00
zhangstar333
c561bdb906 [cherry-pick](branch-21) fix first/last value return error with ignore null (#44996) (#45486) 
cherry-pick from master https://github.com/apache/doris/pull/44996
 2024-12-17 10:04:02 +08:00
github-actions[bot]
23c5d52b04 branch-2.1: [fix](s3) improve error msg #45360 (#45432) 
Cherry-picked from #45360

Co-authored-by: Socrates <suyiteng@selectdb.com>
 2024-12-16 14:59:08 +08:00
github-actions[bot]
1db78d4496 branch-2.1: [fix](hive) fix block decompressor bug #45289 (#45379) 
Cherry-picked from #45289

Co-authored-by: Socrates <suyiteng@selectdb.com>
 2024-12-14 19:20:55 -08:00
meiyi
6dabdd6a8c branch-2.1: [fix](txn insert) txn insert show error url #45254 (#45381) 
Cherry-picked from https://github.com/apache/doris/pull/45254
 2024-12-13 17:36:02 +08:00
wangbo
384a1a4bd7 [branch-2.1]log more detail when query hits policy (#44685) (#45366) 
pick #44685
 2024-12-13 10:24:53 +08:00
wuwenchi
667f5e6e6a [feat](iceberg)Supports using rest type catalog to read tables in unity catalog for 2.1 (#43525) (#45217) 
bp: #43525
 2024-12-12 00:49:36 -08:00
bobhan1
0c55edd3bf branch-2.1: [opt](compaction) Don't check missed rows in cumu compaction if input rowsets are not in tablet (#45279) (#45304) 
pick https://github.com/apache/doris/pull/45279
 2024-12-12 11:14:30 +08:00
github-actions[bot]
da885691ca branch-2.1: [fix](column_array)fix filter_generic in column_array func #41990 (#44983) 
Cherry-picked from #41990

Co-authored-by: amory <wangqiannan@selectdb.com>
 2024-12-12 09:52:26 +08:00
github-actions[bot]
9a4f63f180 branch-2.1: [fix](Outfile) fix the unclear error code #45119 (#45277) 
Cherry-picked from #45119

Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
 2024-12-12 09:32:26 +08:00
shee
fb407f2e94 [opt](lambda) let lambda expression support refer outer slot (#45186) 2024-12-11 18:55:49 +08:00
github-actions[bot]
44fa86040f branch-2.1: [Enhancement](compaction)Optimize compaction task permit allocation #45197 (#45243) 
Cherry-picked from #45197

Co-authored-by: abmdocrt <lianyukang@selectdb.com>
 2024-12-11 14:35:07 +08:00
abmdocrt
d0737c45f8 [cherry-pick](branch-2.1) Pick "[Fix](Rowset Id) Use a randomly generated rowset ID to handle memory write failures (#42949)" (#44086) 2024-12-10 15:48:02 +08:00
Mingyu Chen (Rayner)
e29d125200 [fix](planner) query should be cancelled if limit reached (#44338) (#45222) 
cherry-pick #44338
 2024-12-09 22:22:24 -08:00
Mingyu Chen (Rayner)
5d3f0a267a [opt](scan) unify the local and remote scan bytes stats for all scanners for 2.1 (#45167) 
pick part of #40493

TODO: not working with s3 reader
 2024-12-10 14:19:19 +08:00
wangbo
0bcb99710b [branch-2.1]Add customStdAllocator for vector/map (#41193) (#45124) 2024-12-09 18:33:51 +08:00
wangbo
977da5ca28 [branch-2.1]count tablet meta's static memory load from disk (#41429) (#45128) 
pick #41429
 2024-12-09 14:44:20 +08:00
Xinyi Zou
17bcc208e9 branch-2.1: [fix](arrow-flight-sql) Revert arrow_flight_sql group from regression test #44992 (#45089) 2024-12-08 11:37:02 +08:00
github-actions[bot]
cf2e2113db branch-2.1: [fix](move-memtable) tolerate non-open streams in close wait #44680 (#45154) 
Cherry-picked from #44680

Co-authored-by: Kaijie Chen <chenkaijie@selectdb.com>
 2024-12-08 11:35:30 +08:00
daidai
b332217584 [enchement](mc)mc catalog append netowrk config (#44194) (#45149) 
bp #44194
 2024-12-07 09:52:19 -08:00
924060929
2543af8d3b [feature](function) support orthogonal_bitmap_expr_calculate & orthogonal_bitmap_expr_calculate_count for nereids (#44991) (#45012) 
support orthogonal_bitmap_expr_calculate & orthogonal_bitmap_expr_calculate_count for nereids
 2024-12-07 15:27:48 +08:00
github-actions[bot]
8f0276809a branch-2.1: [fix](compaction) tablet is not removed in compaction queue if dropped #45085 (#45136) 
Cherry-picked from #45085

Co-authored-by: meiyi <meiyi@selectdb.com>
 2024-12-07 12:15:20 +08:00
Mingyu Chen (Rayner)
d4a6fd1850 Revert #43255 & #44615 (#45096) 
Revert "branch-2.1: [enhance](orc) Optimize ORC Predicate Pushdown for
OR-connected Predicate #43255 (#44438)"
Revert "[fix](orc) check all the cases before build_search_argument
(#44615) (#44801)"
 2024-12-06 21:14:13 +08:00
bobhan1
ebb21ef031 branch-2.1: [Fix](merge-on-write) Add defensive check before partial update #44687 (#45086) 
pick https://github.com/apache/doris/pull/44687
 2024-12-06 17:16:42 +08:00
github-actions[bot]
08c8a416ea branch-2.1: [fix](runtime_profile) fix race condition in to_thrift #45047 (#45099) 
Cherry-picked from #45047

Co-authored-by: Kaijie Chen <chenkaijie@selectdb.com>
 2024-12-06 16:25:33 +08:00
Socrates
53291bd040 branch-2.1: [fix](hudi) upgrade hudi to 0.15.0 (#44267)(#44995) (#45041) 
cherry-pick pr: #44267 #44995
 2024-12-05 23:18:37 -08:00
github-actions[bot]
f0324e2a56 branch-2.1: [improvement](information_schema)Support show default value in information_schema. #44849 (#45080) 
Cherry-picked from #44849

Co-authored-by: James <lijibing@selectdb.com>
 2024-12-06 14:54:09 +08:00
github-actions[bot]
94746b764d branch-2.1: [fix](memory) Fix UT ThreadMemTrackerMgrTest #44147 (#44936) 
Cherry-picked from #44147

Co-authored-by: Xinyi Zou <zouxinyi@selectdb.com>
 2024-12-06 10:54:12 +08:00
Mryange
21e21f5e3b [opt](exec) Use PASSTHROUGH to improve the concurrency of the ADAPTIV… (#44971) 
…E_PASSTHROUGH SINK. (#44925)
https://github.com/apache/doris/pull/44925
before
```
op -> local sink(1) -> local source (n)
```
now
```
op -> local passthrough(1) -> local passthrough(n) ->  local sink(n) -> local source (n)
```

profile
```
                  Pipeline  :  1(instance_num=3):
                      AGGREGATION_SINK_OPERATOR  (id=4  ,  nereids_id=255):
                          CROSS_JOIN_OPERATOR  (id=3  ,  nereids_id=245):
                              LOCAL_EXCHANGE_OPERATOR  (ADAPTIVE_PASSTHROUGH)  (id=-5):
                  Pipeline  :  2(instance_num=3):
                      LOCAL_EXCHANGE_SINK_OPERATOR  (ADAPTIVE_PASSTHROUGH)  (id=-5):
                          LOCAL_EXCHANGE_OPERATOR  (PASSTHROUGH)  (id=-6):
                  Pipeline  :  3(instance_num=1):
                      LOCAL_EXCHANGE_SINK_OPERATOR  (PASSTHROUGH)  (id=-6):
                          OLAP_SCAN_OPERATOR  (id=2.  nereids_id=234.  table  name  =  nums1(nums1)):
```
 2024-12-06 09:57:31 +08:00
daidai
0c3bc1bcd0 [fix](auth)fix be enable http auth, some request link never return. (#44959) (#45063) 
bp #44959

if you `enable_all_http_auth = true` in be.conf, then restart be, and
keep using `curl -u "xxxx:xxxx" http://127.0.0.1:8040/api/health` while
be is starting. You may encounter a situation where the link does not
return.
Reason:
When be is still starting, there is no information about fe master. When
you make an api request to be http port, be needs to request
authentication information from fe, which will cause it to request a
machine with empty ip and port 0. This rpc call will definitely fail
(this is not equivalent to a password error). After receiving this
failure, be does not `send_reply` to the api requester, so this api
request cannot be returned.
 2024-12-06 09:39:13 +08:00
github-actions[bot]
a98cb8c8bf branch-2.1: [fix](arrow-flight-sql) Fix query result is empty and not return query error message #45023 (#45053) 
Cherry-picked from #45023

Co-authored-by: Xinyi Zou <zouxinyi@selectdb.com>
 2024-12-06 09:27:35 +08:00
Gabriel
405b50b1b7 [Improvement](queue) Return value of concurrent queue should be proce… (#45032) 
…… (#44986)

…ssed

Push items into concurrent queue will return false due to some
unexpected error (e.g. poor memory available).
 2024-12-05 17:46:18 +08:00