Commit Graph

21515 Commits

Author SHA1 Message Date
cd3e1dce74 [feature](inverted index) Add profile statistics for each condition in inverted index filters (#48459)
https://github.com/apache/doris/pull/47504
2025-03-01 11:00:19 +08:00
0dd532f487 branch-2.1:[fix](nereids)Support negative partition range value and negative column default value. (#48297) (#48480)
backport: https://github.com/apache/doris/pull/48297
2025-03-01 09:08:57 +08:00
4979d70612 branch-2.1: [fix](nereids) Add unique id to non foldable expression #48103 (#48456)
cherry pick from #48103
2025-03-01 08:51:29 +08:00
c6489c1f04 branch-2.1: [fix](regression)Turn auto analyze off before run analyze test. #48414 (#48486)
Cherry-picked from #48414

Co-authored-by: James <lijibing@selectdb.com>
2025-03-01 08:48:40 +08:00
53893c6cb0 branch-2.1: [fix](set variable)Set local session variable when set global variable on Observer. #48468 (#48499)
Cherry-picked from #48468

Co-authored-by: James <lijibing@selectdb.com>
2025-03-01 08:48:00 +08:00
c5e67bf82d branch-2.1: [Chore](client) Do not log in thrift exception when ADDRESS_SANITIZER is defined #48430 (#48455)
Cherry-picked from #48430

Co-authored-by: Pxl <xl@selectdb.com>
2025-02-28 15:12:12 +08:00
d0216690b2 [chore](ci) change s3Source to internal (#48427) (#48471)
Issue Number: close #xxx

Related PR: pick #48427
2025-02-28 15:01:32 +08:00
5a5e433560 branch-2.1:[fix](Outfile) Set retryTime to one while executing outfile statement. (#48423)
Problem Summary:
picked from #48095
2025-02-28 13:57:10 +08:00
5720969a9b [branch-2.1](regression-test) delete case about spark hudi jni reader (#48419) 2025-02-27 22:11:42 +08:00
da90fba590 branch-2.1: [fix](client) Do not log in thrift exception when ADDRESS_SANITIZER is defined #48347 (#48404)
cherry pick from #48347
2025-02-27 15:52:39 +08:00
dae9d9d5e4 [cherry-pick](branch-2.1) Don't prematurely erase DeleteRows in reading iceberg table with position delete (#47977) (#48308)
### What problem does this PR solve?
Issue Number: close #41460
Problem Summary:
When reading the Iceberg table, previously read `DeleteRows` should not
be released immediately, as the Iceberg data file is split into multiple
`IcebergSplit`s for execution. These `IcebergSplit`s belong to the same
data file, meaning they share the same `DeleteRows`. Therefore,
`DeleteRows` in the `DeleteFile` should not be released prematurely.
Instead, they should be released when the shared_kv is reset, at which
point all `DeleteRows` will be freed along with the cached `DeleteFile`.

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-02-27 15:44:40 +08:00
5f35bdf5ba branch-2.1: [fix](backup) make snapshot task with unique signature #48238 (#48303)
cherry pick from #48238
2025-02-27 15:37:03 +08:00
bd8332e769 branch-2.1: [fix](Export) modify some cases of export feature (#48327) 2025-02-27 15:36:16 +08:00
393cb00174 [fix](metrics) max_compaction_score metrics do not update while compaction_num_per_round > 1 (#48383)
refer to #46160
2025-02-27 11:07:15 +08:00
3c41ecbfde branch-2.1: [test](jdbc table) fix jdbc table view test #48236 (#48276)
Cherry-picked from #48236

Co-authored-by: zy-kkk <zhongyk10@gmail.com>
2025-02-27 10:37:18 +08:00
1ed38c1d57 [enhance](auth)The priority of attributes is higher than session vari… (#47877)
…able (#47185)

pick: https://github.com/apache/doris/pull/47185
2025-02-27 10:25:57 +08:00
89b1c2e150 branch-2.1: [fix](nereids) fix merge project contains non foldable expression #48321 (#48366)
cherry pick from #48321
2025-02-27 10:04:08 +08:00
676b868d99 branch-2.1:[opt](docker) Add ranger docker component (#47697) (#48359)
### What problem does this PR solve?
bp  https://github.com/apache/doris/pull/47697

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [x] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [x] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [x] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-02-27 09:47:25 +08:00
6da61683d4 [fix](deps) upgrade hadoop to 3.3.6.5 (#48280) (#48393)
bp #48280
2025-02-26 22:57:50 +08:00
45ebb21cad branch-2.1-pick: [Opt](partial update) Add some cases for partial update (#47900) (#48272)
pick https://github.com/apache/doris/pull/47900
2025-02-26 22:04:57 +08:00
d0cb4f8864 branch-2.1: [fix](schema scan) Fix invalid pointer access #48370 (#48387)
Cherry-picked from #48370

Co-authored-by: Gabriel <liwenqiang@selectdb.com>
2025-02-26 22:04:05 +08:00
d6a5cbb772 branch-2.1: [fix](maxcompute)fix maxcompute partition column schema order #48325 (#48373)
Cherry-picked from #48325

Co-authored-by: daidai <changyuwei@selectdb.com>
2025-02-26 20:34:23 +08:00
11d8030e72 branch-2.1: [Fix](remote-fs)Change closed Field to Instance-Level to Avoid Global Shutdown Issues in RemoteFileSystem #48351 (#48379)
Cherry-picked from #48351

Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>
2025-02-26 20:31:43 +08:00
1d7a5a7b17 branch-2.1: [fix](auth)ldap template ignore partial result exception #47858 (#48358)
Cherry-picked from #47858

Co-authored-by: zhangdong <zhangdong@selectdb.com>
2025-02-26 20:30:12 +08:00
4f3ccca9a2 branch-2.1: [fix](schema scan) Fix invalid pointer access #48313 (#48341)
Cherry-picked from #48313

Co-authored-by: Gabriel <liwenqiang@selectdb.com>
2025-02-26 20:29:39 +08:00
a5adb8ec69 branch-2.1: [fix](partition) Incorrectly add partition to non-partitioned table #48121 (#48323)
Cherry-picked from #48121

Co-authored-by: Uniqueyou <wangyixuan@selectdb.com>
2025-02-26 20:28:45 +08:00
b1ac39587b [fix](variant) fix variant used in order by back to legacy planner would meet core (#48332)
### What problem does this PR solve?
sql 
```
 SELECT    *,    ROW_NUMBER() OVER (     PARTITION BY orderid      ORDER BY v DESC   ) AS row_idfirst  FROM test_v_legacy order by orderid;
```
core:
```
F20250225 11:23:08.735848 1788347 column_object.h:439] should not call the method in column object
*** Check failure stack trace: ***
F20250225 11:23:08.736686 1788345 column_object.h:439] should not call the method in column object
*** Check failure stack trace: ***
    @     0x5555fa0821b6  google::LogMessage::SendToLog()
    @     0x5555fa0821b6  google::LogMessage::SendToLog()
    @     0x5555fa07ec00  google::LogMessage::Flush()
    @     0x5555fa07ec00  google::LogMessage::Flush()
    @     0x5555fa0829f9  google::LogMessageFatal::~LogMessageFatal()
    @     0x5555fa0829f9  google::LogMessageFatal::~LogMessageFatal()
    @     0x5555dd722b1c  doris::vectorized::ColumnObject::get_permutation()
    @     0x5555dd722b1c  doris::vectorized::ColumnObject::get_permutation()
    @     0x5555dd6b933c  doris::vectorized::ColumnNullable::get_permutation()
    @     0x5555dd6b933c  doris::vectorized::ColumnNullable::get_permutation()
    @     0x5555dde3997f  doris::vectorized::sort_block()
    @     0x5555dde3997f  doris::vectorized::sort_block()
    @     0x5555e2f13646  doris::vectorized::Sorter::partial_sort()
    @     0x5555e2f13646  doris::vectorized::Sorter::partial_sort()
    @     0x5555e2f1503b  doris::vectorized::FullSorter::_do_sort()
    @     0x5555e2f1503b  doris::vectorized::FullSorter::_do_sort()
    @     0x5555e2f15a2f  doris::vectorized::FullSorter::prepare_for_read()
    @     0x5555e2f15a2f  doris::vectorized::FullSorter::prepare_for_read()
    @     0x5555f97fb76a  doris::pipeline::SortSinkOperatorX::sink()
    @     0x5555f97fb76a  doris::pipeline::SortSinkOperatorX::sink()
    @     0x5555fa018798  doris::pipeline::PipelineXTask::execute()
    @     0x5555fa018798  doris::pipeline::PipelineXTask::execute()
    @     0x5555fa04efe5  doris::pipeline::TaskScheduler::_do_work()
    @     0x5555fa04efe5  doris::pipeline::TaskScheduler::_do_work()
    @     0x5555fa052f2b  doris::pipeline::TaskScheduler::start()::$_0::operator()()
    @     0x5555fa052f2b  doris::pipeline::TaskScheduler::start()::$_0::operator()()

```
Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [x] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-02-26 20:27:43 +08:00
82fe8bc3d7 branch-2.1:[fix](libhdfs) fix the lifecycle issue of libhdfs config (#48352)
pick part of #47299
when calling `hdfsBuilderSetKerb5Conf`, the `value` string's lifecycle
must be with `hdfs_builder`.
2025-02-26 14:10:49 +08:00
69196d8212 [fix](jvm) the jvm opt should only be set once #48335 (#48336)
bp #48335
2025-02-26 09:52:58 +08:00
a2fe1bda7f branch-2.1: [fix](group commit) fix wal reader handle empty block #48290 (#48334)
Cherry-picked from #48290

Co-authored-by: meiyi <meiyi@selectdb.com>
2025-02-26 09:51:55 +08:00
babab64bbd branch-2.1: [fix](group commit) group commit failed if enable global enable_unique_key_partial_update (#48251)
pick https://github.com/apache/doris/pull/48120
2025-02-25 22:08:50 +08:00
4f8ff321ea branch-2.1: [fix](inverted index) fix wrong read data for primary key #47841 (#48207)
Cherry-picked from #47841

Co-authored-by: airborne12 <jiangkai@selectdb.com>
2025-02-25 21:15:03 +08:00
d6f4db4170 [fix](load) fix metrics memtable_flush_duration_us (#47607) (#48213)
backport #47607
2025-02-25 21:13:03 +08:00
e63df94c9d [fix](cases) fix unstable leading cases (#46441) (#47896) 2025-02-25 20:38:34 +08:00
65e9dce3da branch-2.1: [fix](Nereids) change numeric arithmatic function boundary behavior to match with be execution behavior #47966 (#48237)
pick:#47966

Related PR: #40744, #47228

Problem Summary:
When numeric function input or output out of boundary when fe folding
constant, it would throw an exception before. Now
we change it to match with be execution result, which is NullLiteral

### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-02-25 20:38:00 +08:00
769eca534d branch-2.1: [fix](test) fix unstable jdbc test case #48223 (#48249)
Cherry-picked from #48223

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
2025-02-25 20:34:25 +08:00
24702bfbf1 [BugFix](PreparedStatement) fix stmtId overflow (#47950) (#48271)
cherry-pick from  (#47950)
2025-02-25 20:33:50 +08:00
a3e16d3db0 branch-2.1: [fix](olap) clear storage_policy property when is_being_synced = true #48229 (#48304)
cherry pick from #48229
2025-02-25 20:33:05 +08:00
Pxl
7a98ac434f [Chore](case) pick some broadcast join case from #48247 #47380 (#48310)
### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-02-25 20:32:12 +08:00
1aa57a3b13 branch-2.1: [fix](array index) Correct null bitmap writing for inverted index #47846 (#48214)
cherry pick from #47846 #48231
2025-02-25 20:31:18 +08:00
27cd8636bb branch-2.1: [fix](audit) fix sql has same sqlhash in one request #48242 (#48270)
Cherry-picked from #48242

Co-authored-by: feiniaofeiafei <moailing@selectdb.com>
2025-02-25 19:07:28 +08:00
af1dcbe17c [cherry-pick](branch-21) fix unstable test case (#48141) (#48274) 2025-02-25 19:06:55 +08:00
0d386f9268 branch-2.1: [fix](java) should use JAVA_OPTS_FOR_JDK_17 instead of JAVA_OPTS #48170 (#48282)
bp #48170
So that when user using jdk17 to run Doris 2.1, it will get the right
JAVA_OPTS
2025-02-25 18:23:18 +08:00
47a1315cee branch-2.1: [fix](Export) Clone SessionVariable for each ExportTaskExecutor #48239 (#48255)
Cherry-picked from #48239

Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
2025-02-25 09:27:05 +08:00
f5de8b0742 branch-2.1: [bug](restore) Fix wrong storage policy in show create stmt after restore #48158 (#48227)
Cherry-picked from #48158

Co-authored-by: xy720 <22125576+xy720@users.noreply.github.com>
2025-02-25 09:26:24 +08:00
ccd1b073f8 branch-2.1: [fix](group commit) check group commit session value (#48253)
pick https://github.com/apache/doris/pull/48112
2025-02-24 21:55:53 +08:00
47695319be branch-2.1: [feat](ci) enable enable_parquet_page_index in pipeline (#48217) (#48250)
bp #48217
2025-02-24 21:01:49 +08:00
3ad9227ea1 branch-2.1: [fix](regression-test) Fix unstable regression test case due to trigger compaction failed #48228 (#48246)
Cherry-picked from #48228

Co-authored-by: Siyang Tang <tangsiyang@selectdb.com>
2025-02-24 20:37:38 +08:00
c41d316114 [feature](metrics) add catalog/db/table num metrics (#47891) (#47998)
bp #47891
2025-02-24 20:35:39 +08:00
0e85e183d1 branch-2.1: [test](external) Add restart_fe tag to test suites #48181 (#48235)
Cherry-picked from #48181

Co-authored-by: zy-kkk <zhongyk10@gmail.com>
2025-02-24 16:36:45 +08:00