Commit Graph

21497 Commits

Author SHA1 Message Date
6da61683d4 [fix](deps) upgrade hadoop to 3.3.6.5 (#48280) (#48393)
bp #48280
2025-02-26 22:57:50 +08:00
45ebb21cad branch-2.1-pick: [Opt](partial update) Add some cases for partial update (#47900) (#48272)
pick https://github.com/apache/doris/pull/47900
2025-02-26 22:04:57 +08:00
d0cb4f8864 branch-2.1: [fix](schema scan) Fix invalid pointer access #48370 (#48387)
Cherry-picked from #48370

Co-authored-by: Gabriel <liwenqiang@selectdb.com>
2025-02-26 22:04:05 +08:00
d6a5cbb772 branch-2.1: [fix](maxcompute)fix maxcompute partition column schema order #48325 (#48373)
Cherry-picked from #48325

Co-authored-by: daidai <changyuwei@selectdb.com>
2025-02-26 20:34:23 +08:00
11d8030e72 branch-2.1: [Fix](remote-fs)Change closed Field to Instance-Level to Avoid Global Shutdown Issues in RemoteFileSystem #48351 (#48379)
Cherry-picked from #48351

Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>
2025-02-26 20:31:43 +08:00
1d7a5a7b17 branch-2.1: [fix](auth)ldap template ignore partial result exception #47858 (#48358)
Cherry-picked from #47858

Co-authored-by: zhangdong <zhangdong@selectdb.com>
2025-02-26 20:30:12 +08:00
4f3ccca9a2 branch-2.1: [fix](schema scan) Fix invalid pointer access #48313 (#48341)
Cherry-picked from #48313

Co-authored-by: Gabriel <liwenqiang@selectdb.com>
2025-02-26 20:29:39 +08:00
a5adb8ec69 branch-2.1: [fix](partition) Incorrectly add partition to non-partitioned table #48121 (#48323)
Cherry-picked from #48121

Co-authored-by: Uniqueyou <wangyixuan@selectdb.com>
2025-02-26 20:28:45 +08:00
b1ac39587b [fix](variant) fix variant used in order by back to legacy planner would meet core (#48332)
### What problem does this PR solve?
sql 
```
 SELECT    *,    ROW_NUMBER() OVER (     PARTITION BY orderid      ORDER BY v DESC   ) AS row_idfirst  FROM test_v_legacy order by orderid;
```
core:
```
F20250225 11:23:08.735848 1788347 column_object.h:439] should not call the method in column object
*** Check failure stack trace: ***
F20250225 11:23:08.736686 1788345 column_object.h:439] should not call the method in column object
*** Check failure stack trace: ***
    @     0x5555fa0821b6  google::LogMessage::SendToLog()
    @     0x5555fa0821b6  google::LogMessage::SendToLog()
    @     0x5555fa07ec00  google::LogMessage::Flush()
    @     0x5555fa07ec00  google::LogMessage::Flush()
    @     0x5555fa0829f9  google::LogMessageFatal::~LogMessageFatal()
    @     0x5555fa0829f9  google::LogMessageFatal::~LogMessageFatal()
    @     0x5555dd722b1c  doris::vectorized::ColumnObject::get_permutation()
    @     0x5555dd722b1c  doris::vectorized::ColumnObject::get_permutation()
    @     0x5555dd6b933c  doris::vectorized::ColumnNullable::get_permutation()
    @     0x5555dd6b933c  doris::vectorized::ColumnNullable::get_permutation()
    @     0x5555dde3997f  doris::vectorized::sort_block()
    @     0x5555dde3997f  doris::vectorized::sort_block()
    @     0x5555e2f13646  doris::vectorized::Sorter::partial_sort()
    @     0x5555e2f13646  doris::vectorized::Sorter::partial_sort()
    @     0x5555e2f1503b  doris::vectorized::FullSorter::_do_sort()
    @     0x5555e2f1503b  doris::vectorized::FullSorter::_do_sort()
    @     0x5555e2f15a2f  doris::vectorized::FullSorter::prepare_for_read()
    @     0x5555e2f15a2f  doris::vectorized::FullSorter::prepare_for_read()
    @     0x5555f97fb76a  doris::pipeline::SortSinkOperatorX::sink()
    @     0x5555f97fb76a  doris::pipeline::SortSinkOperatorX::sink()
    @     0x5555fa018798  doris::pipeline::PipelineXTask::execute()
    @     0x5555fa018798  doris::pipeline::PipelineXTask::execute()
    @     0x5555fa04efe5  doris::pipeline::TaskScheduler::_do_work()
    @     0x5555fa04efe5  doris::pipeline::TaskScheduler::_do_work()
    @     0x5555fa052f2b  doris::pipeline::TaskScheduler::start()::$_0::operator()()
    @     0x5555fa052f2b  doris::pipeline::TaskScheduler::start()::$_0::operator()()

```
Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [x] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-02-26 20:27:43 +08:00
82fe8bc3d7 branch-2.1:[fix](libhdfs) fix the lifecycle issue of libhdfs config (#48352)
pick part of #47299
when calling `hdfsBuilderSetKerb5Conf`, the `value` string's lifecycle
must be with `hdfs_builder`.
2025-02-26 14:10:49 +08:00
69196d8212 [fix](jvm) the jvm opt should only be set once #48335 (#48336)
bp #48335
2025-02-26 09:52:58 +08:00
a2fe1bda7f branch-2.1: [fix](group commit) fix wal reader handle empty block #48290 (#48334)
Cherry-picked from #48290

Co-authored-by: meiyi <meiyi@selectdb.com>
2025-02-26 09:51:55 +08:00
babab64bbd branch-2.1: [fix](group commit) group commit failed if enable global enable_unique_key_partial_update (#48251)
pick https://github.com/apache/doris/pull/48120
2025-02-25 22:08:50 +08:00
4f8ff321ea branch-2.1: [fix](inverted index) fix wrong read data for primary key #47841 (#48207)
Cherry-picked from #47841

Co-authored-by: airborne12 <jiangkai@selectdb.com>
2025-02-25 21:15:03 +08:00
d6f4db4170 [fix](load) fix metrics memtable_flush_duration_us (#47607) (#48213)
backport #47607
2025-02-25 21:13:03 +08:00
e63df94c9d [fix](cases) fix unstable leading cases (#46441) (#47896) 2025-02-25 20:38:34 +08:00
65e9dce3da branch-2.1: [fix](Nereids) change numeric arithmatic function boundary behavior to match with be execution behavior #47966 (#48237)
pick:#47966

Related PR: #40744, #47228

Problem Summary:
When numeric function input or output out of boundary when fe folding
constant, it would throw an exception before. Now
we change it to match with be execution result, which is NullLiteral

### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-02-25 20:38:00 +08:00
769eca534d branch-2.1: [fix](test) fix unstable jdbc test case #48223 (#48249)
Cherry-picked from #48223

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
2025-02-25 20:34:25 +08:00
24702bfbf1 [BugFix](PreparedStatement) fix stmtId overflow (#47950) (#48271)
cherry-pick from  (#47950)
2025-02-25 20:33:50 +08:00
a3e16d3db0 branch-2.1: [fix](olap) clear storage_policy property when is_being_synced = true #48229 (#48304)
cherry pick from #48229
2025-02-25 20:33:05 +08:00
Pxl
7a98ac434f [Chore](case) pick some broadcast join case from #48247 #47380 (#48310)
### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-02-25 20:32:12 +08:00
1aa57a3b13 branch-2.1: [fix](array index) Correct null bitmap writing for inverted index #47846 (#48214)
cherry pick from #47846 #48231
2025-02-25 20:31:18 +08:00
27cd8636bb branch-2.1: [fix](audit) fix sql has same sqlhash in one request #48242 (#48270)
Cherry-picked from #48242

Co-authored-by: feiniaofeiafei <moailing@selectdb.com>
2025-02-25 19:07:28 +08:00
af1dcbe17c [cherry-pick](branch-21) fix unstable test case (#48141) (#48274) 2025-02-25 19:06:55 +08:00
0d386f9268 branch-2.1: [fix](java) should use JAVA_OPTS_FOR_JDK_17 instead of JAVA_OPTS #48170 (#48282)
bp #48170
So that when user using jdk17 to run Doris 2.1, it will get the right
JAVA_OPTS
2025-02-25 18:23:18 +08:00
47a1315cee branch-2.1: [fix](Export) Clone SessionVariable for each ExportTaskExecutor #48239 (#48255)
Cherry-picked from #48239

Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
2025-02-25 09:27:05 +08:00
f5de8b0742 branch-2.1: [bug](restore) Fix wrong storage policy in show create stmt after restore #48158 (#48227)
Cherry-picked from #48158

Co-authored-by: xy720 <22125576+xy720@users.noreply.github.com>
2025-02-25 09:26:24 +08:00
ccd1b073f8 branch-2.1: [fix](group commit) check group commit session value (#48253)
pick https://github.com/apache/doris/pull/48112
2025-02-24 21:55:53 +08:00
47695319be branch-2.1: [feat](ci) enable enable_parquet_page_index in pipeline (#48217) (#48250)
bp #48217
2025-02-24 21:01:49 +08:00
3ad9227ea1 branch-2.1: [fix](regression-test) Fix unstable regression test case due to trigger compaction failed #48228 (#48246)
Cherry-picked from #48228

Co-authored-by: Siyang Tang <tangsiyang@selectdb.com>
2025-02-24 20:37:38 +08:00
c41d316114 [feature](metrics) add catalog/db/table num metrics (#47891) (#47998)
bp #47891
2025-02-24 20:35:39 +08:00
0e85e183d1 branch-2.1: [test](external) Add restart_fe tag to test suites #48181 (#48235)
Cherry-picked from #48181

Co-authored-by: zy-kkk <zhongyk10@gmail.com>
2025-02-24 16:36:45 +08:00
feb4b09cb3 [fix](hive)Spelling mistake of the word "failed" for 2.1 (#48193) 2025-02-22 23:18:21 +08:00
8ffd6b8c3e branch-2.1: [case](mtmv)The impact of validating the base table and materialized view alter on transparent rewriting #47310 (#48186)
Cherry-picked from https://github.com/apache/doris/pull/47310
2025-02-22 23:12:24 +08:00
cc0339a6ab branch-2.1: [case](mtmv)MTMV upgrade downgrade case #39789 (#48185)
Cherry-picked from #39789

Co-authored-by: zhangdong <493738387@qq.com>
2025-02-22 13:33:08 +08:00
514d419c19 branch-2.1: [fix](binlog) Binlog ts not initialized #47174 (#48067)
pick: https://github.com/apache/doris/pull/47174
2025-02-21 20:23:16 +08:00
bcd01c5f24 branch-2.1-pick: [Fix](merge-on-write) should calculate delete bitmaps between segmens before skip if tablet is in NOT_READY state in flush phase (#48056) (#48089)
pick https://github.com/apache/doris/pull/48056
2025-02-21 19:53:17 +08:00
469bc77a42 branch-2.1: [fix](jdbc catalog) Change BE jdbc Driver loading to Java code (#48002)
cherry-pick from (#46912)
2025-02-21 17:39:26 +08:00
0fe6c17ce2 branch-2.1: [fix](restore) Remove dynamic_partition.storage_policy property #48134 (#48160)
Cherry-picked from #48134

Co-authored-by: walter <maochuan@selectdb.com>
2025-02-21 17:37:37 +08:00
a3753c7edb [opt](nereids) optimize not found function error message (#47919) (#48102)
cherry pick from #47919
2025-02-21 10:13:46 +08:00
3738d0fddc branch-2.1: [fix](nereids) fix push down non-foldable filter through project #47989 (#48084)
cherry pick from #47989
2025-02-21 10:09:03 +08:00
4d5c337b23 branch-2.1: [feature](restore) support force_replace restore#47314 (#48051)
pick: https://github.com/apache/doris/pull/47314
2025-02-21 08:34:55 +08:00
3fa4413e3b branch-2.1: [fix](restore) Lock tablet before modify segment files #45711 (#48047)
cherry pick from #45711
2025-02-21 08:33:10 +08:00
46c2a0493c branch-2.1: [fix](paimon) avoid get paimon table when replay (#48151)
### What problem does this PR solve?

`catalog.getPaimonTable(dbName, name);` will try forward request to
master FE.
When in replay logic, forwarding will fail, cause fe failed to start.
Introduced from #43959
2025-02-21 01:07:55 +08:00
360933b885 branch-2.1: [feat](binlog) Speed binlog gc by locked binlogs #47547 (#48128)
cherry pick from #47547
2025-02-20 17:08:26 +08:00
7f65836d9a branch-2.1: [fix](nereids)check if correlated filter exists before converting apply to join #47985 (#48091)
Cherry-picked from #47985

Co-authored-by: starocean999 <lichi@selectdb.com>
2025-02-20 16:29:06 +08:00
c5d4b1173d branch-2.1: [enhance](mtmv)Mtmv support audit log #41499 (#47923)
pick from master #41499
2025-02-20 14:01:54 +08:00
bb6641b63e [cherry-pick](branch-21) add more signatures for lag/lead fucntion (#47940) (#48076) 2025-02-19 20:41:40 +08:00
caad4a7ff1 branch-2.1: [fix](table) Filter base idx id for get sycned table ddl #47970 (#48065)
pick: https://github.com/apache/doris/pull/47970
2025-02-19 19:26:50 +08:00
ae0779641c branch-2.1: [feat](binlog) Add lock binlog method #46887 (#48046)
cherry pick from #46887
2025-02-19 19:24:37 +08:00