Commit Graph

10020 Commits

Author SHA1 Message Date
9fe87a56bd [cherry-pick](branch-21) support lead/lag function input column as third params (#49381) (#50653)
### What problem does this PR solve?

cherry-pick from master (#49381)

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-05-07 23:08:56 +08:00
b9cfdcf46a branch-2.1: [chore](dependency)bump tomcat 8.5.86 to 9.0.104 #50622 (#50645)
Cherry-picked from #50622

Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>
2025-05-07 23:06:46 +08:00
22d846423d [fix](paimon)Set the target size of the split for 2.1 (#50596)
### What problem does this PR solve?

bp: #50083
2025-05-07 23:05:10 +08:00
6c62f28499 branch-2.1: [fix](RecycleBin) fix race condition issue when do execute ShowCatalogRecycleBinStmt (#35759) (#50654)
bp #35759

Co-authored-by: caiconghui <55968745+caiconghui@users.noreply.github.com>
Co-authored-by: caiconghui1 <caiconghui1@jd.com>
2025-05-07 23:02:16 +08:00
e52511ba1b branch-2.1: [chore](test) add ut for sql cache #49389 (#49549)
Cherry-picked from #49389

Co-authored-by: 924060929 <lanhuajian@selectdb.com>
2025-05-07 18:55:41 +08:00
e1c50c8e39 branch-2.1: [fix](mtmv)The change to the max_persistence_task_count configuration applies retroactively to existing MTMV #50537 (#50627)
Cherry-picked from #50537

Co-authored-by: zhangdong <zhangdong@selectdb.com>
2025-05-07 12:07:40 +08:00
a066374f94 [opt](paimon/iceberg)upgrade paimon to 1.0.1, iceberg to 1.6.1 for 2.1 (#50612)
bp #46990 #47768 #47117

Co-authored-by: Socrates <suyiteng@selectdb.com>
Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
2025-05-07 09:31:20 +08:00
1de9e7f9e8 branch-2.1: [fix](view)fix reset view def for restore wrong replace #50567 (#50600)
Cherry-picked from #50567

Co-authored-by: koarz <lihao@selectdb.com>
2025-05-06 20:15:45 +08:00
c3812b382b branch-2.1: [fix](nereids) fix bug of MergePercentileToArray #49773 (#50603)
Cherry-picked from #49773

Co-authored-by: feiniaofeiafei <moailing@selectdb.com>
2025-05-06 17:09:32 +08:00
a9b948f476 branch-2.1: [fix](catalog) do cache load when cache value is not present #50188 (#50451)
Cherry-picked from #50188

---------

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
Co-authored-by: morningman <yunyou@selectdb.com>
2025-05-06 10:59:11 +08:00
65151b46aa branch-2.1: [improve](routine load) add more metrics to observe the routine load job #48209 (#48765)
Cherry-picked from #48209

Co-authored-by: hui lai <laihui@selectdb.com>
2025-05-05 10:25:37 +08:00
98be2cedcf branch-2.1: [fix](constant fold)Make sure FE cast double to varchar generate identical result with BE. #50425 (#50548)
Cherry-picked from #50425

Co-authored-by: James <lijibing@selectdb.com>
2025-04-30 09:17:47 +08:00
2b62fb1a59 [fix](audit) fix potential audit log missing issue (#50357) (#50501)
bp #50357
2025-04-29 08:27:13 +08:00
fcc05d630b [chore](show engines)Change palo to Doris (#47861) (#50459)
bp #47861
2025-04-28 18:57:56 +08:00
d23bd87247 branch-2.1: [fix](txn) Restart fe after commit txn causes loadedTblIndexes lossing #50123 (#50206)
Cherry-picked from #50123

Co-authored-by: Uniqueyou <wangyixuan@selectdb.com>
2025-04-26 14:58:36 +08:00
90e22197ff [fix](json-functions)fix json-replace/insert/set/array behavior with complex type #50308 (#50309)
### What problem does this PR solve?
backport: https://github.com/apache/doris/pull/50308

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-04-24 19:04:25 +08:00
dd43f13549 branch-2.1:[fix](ut)Fix StmtExecutorTest (#50322) (#50362)
backport: https://github.com/apache/doris/pull/50322
2025-04-24 16:18:02 +08:00
91eed373be branch-2.1: [fix](array_map) fix array_map functions avoid core #50201 (#50331)
Cherry-picked from #50201

---------

Co-authored-by: amory <wangqiannan@selectdb.com>
2025-04-24 15:28:12 +08:00
cf72fa82e2 [Improve](explode) explode function support multi param (#50310)
### What problem does this PR solve?
backport:https://github.com/apache/doris/pull/48537
Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-04-23 23:27:07 +08:00
b0c8cb0818 branch-2.1: [Bug](function) fix Could not find function explode_json_array_json_outer #50164 (#50227)
Cherry-picked from #50164

Co-authored-by: zhangstar333 <zhangsida@selectdb.com>
2025-04-23 15:59:08 +08:00
98f5d95fba branch-2.1: [feat](function) SUBSTRING_INDEX function delimiter supports dynamic #50149 (#50303)
Cherry-picked from #50149

Co-authored-by: lw112 <131352377+felixwluo@users.noreply.github.com>
2025-04-23 15:00:40 +08:00
a193f16a5a branch-2.1:[fix](auditlog)Set sqlHash in executeInternalQuery (#49984) (#50271)
backport: https://github.com/apache/doris/pull/49984
2025-04-22 18:05:56 +08:00
1efe62c7ba branch-2.1: [opt](hive) add option to get schema from table object #50038 (#50269)
Cherry-picked from #50038

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
2025-04-22 14:25:03 +08:00
b9bb9e1611 branch-2.1: [fix](Export) fix the lock leak issue of Export #50088 (#50219)
Cherry-picked from #50088

Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
2025-04-22 14:21:42 +08:00
1c90fc6c02 branch-2.1: [fix](connect) fix wrong format causing connection fail #50214 (#50217)
bp #50214
2025-04-21 16:34:00 +08:00
188cbabb58 [enhance](auth)Remove restrictions on user creation and other operations when enabling ranger/LDAP (#50137)
### What problem does this PR solve?
- In version 2.1, the global permission check still calls the internal
permission interface. If grant is not allowed, it will be impossible to
assign admin and other permissions to users
- According to the current design of LDAP, if there is no user in LDAP,
Doris will check again to see if the user exists internally. If there
is, login will also be allowed. Therefore, creating users should not be
prohibited
2025-04-21 14:15:07 +08:00
d575a0921a [fix](hive) Fix hive metastore client socket timeout (#50192) 2025-04-21 14:09:20 +08:00
01b5cc4f1f [opt](mtmv) Doesn't throw npe when not set distribution info which should be random (#49402)
### What problem does this PR solve?

pr: https://github.com/apache/doris/pull/39427
commitId: 9ffb060a



Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-04-21 10:41:07 +08:00
481fca6766 [revert](statistics)Still use sample analyze sql when sample full table. (#50199)
Still use sample analyze sql when sample full table.
2025-04-21 10:12:01 +08:00
048537cd03 branch-2.1:[fix](nereids)Use utf-8 when convert string like literal to double. (#50085) (#50155)
backport: https://github.com/apache/doris/pull/50085
2025-04-19 17:16:20 +08:00
967d0a59b9 branch-2.1: [fix](load) reset routine load task EOF sign correctly #50048 (#50179)
Cherry-picked from #50048

Co-authored-by: hui lai <laihui@selectdb.com>
2025-04-19 17:15:18 +08:00
64815c6403 branch-2.1: [fix](dialect)fix starrocks dialect #50099 (#50165)
Cherry-picked from #50099

Co-authored-by: Jiwen liu <61498169+liujiwen-up@users.noreply.github.com>
2025-04-18 20:25:38 +08:00
4146cbbbbb branch-2.1-pick: [Opt](mow) Forbid time_series compaction policy on unique table (#49905) (#50132)
pick https://github.com/apache/doris/pull/49905
2025-04-18 11:34:18 +08:00
77ea907b54 branch-2.1: [Bug](materialized-view) Fixed the problem of using drop table force and create mv stmt at the… #41580 (#50133)
Cherry-picked from #41580

Co-authored-by: Pxl <pxl290@qq.com>
2025-04-18 11:30:47 +08:00
ea29bc523e branch-2.1: [Enhancement](GEO) Support Multipolygon and some spatial functions (#50073)
pick: https://github.com/apache/doris/pull/37003,
https://github.com/apache/doris/pull/48695 and
https://github.com/apache/doris/pull/49665

---------

Co-authored-by: Mryange <59914473+Mryange@users.noreply.github.com>
Co-authored-by: koi <koi20000@163.com>
2025-04-17 09:25:29 +08:00
2ef136cad6 [fix](variant) building index on the variant column is prohibited (#49159) (#50028)
pick from master #49159
2025-04-16 17:52:56 +08:00
bbdb057e2c branch-2.1: [fix](nereids) EliminateGroupByConstant should replace agg's group by after removing constant group by keys #49473 (#50044)
Cherry-picked from #49473

Co-authored-by: feiniaofeiafei <moailing@selectdb.com>
2025-04-16 13:02:39 +08:00
7ba2f87087 branch-2.1: [fix](binlog) Replaced partition should be record to Dropped Resources #48686 (#49789)
pick: https://github.com/apache/doris/pull/48686/files
2025-04-16 12:41:12 +08:00
b9d447741e Revert "branch-2.1: [Fix](Serde) Support hive compatible output format #49036" (#49986)
Reverts apache/doris#49831
2025-04-11 16:08:02 +08:00
9960b0960e branch-2.1: [bug](auto partition) Fix be crash with single replica insert (#48929)
Something may be wrong in #48536, and it reverts at #48926.

We need to re-run the pipeline and see what was wrong.
2025-04-11 15:23:51 +08:00
ba1ad12a21 branch-2.1: [Fix](Serde) Support hive compatible output format #49036 (#49831)
Cherry-picked from #49036

Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
2025-04-11 15:13:02 +08:00
40a62c3b82 branch-2.1: [feat](dialect) add starrocks dialect #49951 (#49974)
Cherry-picked from #49951

Co-authored-by: Jiwen liu <61498169+liujiwen-up@users.noreply.github.com>
2025-04-11 09:23:06 +08:00
530bad3d14 Set isQuery audit log correctly when fallback to old planner. (#49959) 2025-04-10 23:27:54 +08:00
78b8f31895 branch-2.1 [Fix](Catalog) Close system resources when dropping catalog ( #49621) (#49935)
https://github.com/apache/doris/pull/49621
(cherry picked from commit
902b95b1bf)
2025-04-10 23:23:51 +08:00
e4d0c672a3 branch-2.1: [fix](Nereids) cse extract wrong expression from lambda expressions (#49166) (#49942)
cherry pick from #49166

Co-authored-by: morrySnow <zhangwenxin@selectdb.com>
2025-04-10 23:22:55 +08:00
c83c2e18bc [fix](planner) fix fallback to legacy planner compute wrong result (#49913)
### What problem does this PR solve?

doris 3.x not need this fix because the legacy planner already be
deleted

what's wrong:
1. when nereids planner can not parse the sql, doris will fallback to
legacy planner and maybe handle this case, but we not set the
`enable_nereids_planner` to `false` for legacy planner
2. the `ignore_storage_data_distribution` only support when
`enable_nereids_planner` is `true`
3. the legacy planner will use one phase aggregation when input fragment
only has one instance
4. the legacy planner maybe compute one instance by invoke
`OlapScanNode#getNumInstances()` if `enable_nereids_planner == true &&
ignore_storage_data_distribution == true`, even the data place at
multiple backends, then the aggregation will compute wrong result

so when fallback to legacy planner, the legacy planner should set
`enable_nereids_planner` to `false`, then the input OlapScanNode will
not compute one instance, and the AggregationNode will use two phases to
compute the right result.


### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [x] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [x] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-04-10 23:22:18 +08:00
bfe7469560 branch-2.1: [fix](oss) the write to hive table on oss-hdfs may fail (#49754) (#49911)
bp #49754
2025-04-10 22:01:42 +08:00
4ae9451b63 branch-2.1: [opt](connection) add connection num in error msg (#49471) (#49910)
bp #49471
2025-04-10 22:00:51 +08:00
81ea088c96 branch-2.1: [fix](backup) Save snapshot meta during replay #49550 (#49606)
Cherry-picked from #49550

Co-authored-by: walter <maochuan@selectdb.com>
2025-04-09 14:19:19 +08:00
55974f7fb9 [fix](mtmv) Fix materialized rewrite oom when the num of relation mapping is too large (#48887) (#49850) 2025-04-08 15:18:26 +08:00