Commit Graph

10033 Commits

Author SHA1 Message Date
79056d4d7a branch-2.1: [feat](hive) add catalog level partition cache property #50724 (#50762)
Cherry-picked from #50724

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
2025-05-09 22:01:49 +08:00
fee5d40e07 [fix](planner) fix show variable display wrong enable_nereids_planner value (#50746)
### What problem does this PR solve?

fix show variable display wrong enable_nereids_planner value, introduced
by #49913

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [x] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [x] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-05-09 17:09:07 +08:00
04cbb2ac66 [fix](information_schema) fix backend_active_tasks table only return one backend's data (#50721) (#50722)
cherry pick from #50721
2025-05-09 15:01:22 +08:00
fde8d05f5d branch-2.1 [opt](nereids) catch all exceptions in StatsCalculator (#49415) (#50364) 2025-05-09 11:24:18 +08:00
0347e8a3c6 branch-2.1: proper planning of shadow columns for load and schema change concurrent execution (#49332) (#50710) 2025-05-09 11:15:08 +08:00
37f3c8f0c7 [fix](nereids) fix fold constant return wrong scale of datetime type (#50142) (#50716)
cherry pick from #50142
2025-05-09 11:12:07 +08:00
bf5885a8f8 branch-2.1: [opt](Nereids) avoid generate nested alias expr when plan insert values (#50388) 2025-05-09 11:06:24 +08:00
12e6a02158 branch-2.1: [enhance](mtmv)Exclude trigger table support db and catalog #49961 (#50554)
Cherry-picked from #49961

Co-authored-by: zhangdong <zhangdong@selectdb.com>
2025-05-09 09:22:31 +08:00
523681d58e branch-2.1: [enhance](mtmv)Only restrict MTMV to not allow concurrent insert overwrite execution #48673 (#49965)
Cherry-picked from #48673

Co-authored-by: zhangdong <zhangdong@selectdb.com>
2025-05-08 19:19:30 +08:00
ebe302cb7e branch-2.1: [fix](nereids) do eliminate constant group by key in normalizeagg #49589 (#50212)
Cherry-picked from https://github.com/apache/doris/pull/49589
2025-05-08 18:52:40 +08:00
995f1e5dc0 branch-2.1:[fix](Nereids) fix regression framework compare issue and fix code point count (#49575) (#50667)
backport: https://github.com/apache/doris/pull/49575

Co-authored-by: LiBinfeng <libinfeng@selectdb.com>
2025-05-08 16:53:02 +08:00
6f84c4d7e2 branch-2.1: [fix](planner) return explicit error msg when falling back to old planner (#50676)
### What problem does this PR solve?

Problem Summary:

When fall back to old planner, the partition pruning of external table
will fail,
and NPE will be throw like:
```
2025-04-14 16:22:33,427 WARN (mysql-nio-pool-19697|237161) [HiveScanNode.getSplits():189] get file split failed for table: dwd_log_fact_channel_track_result_hi
java.lang.NullPointerException: null
        at org.apache.doris.datasource.hive.source.HiveScanNode.getPartitions(HiveScanNode.java:137) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.datasource.hive.source.HiveScanNode.getSplits(HiveScanNode.java:171) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.datasource.FileQueryScanNode.createScanRangeLocations(FileQueryScanNode.java:366) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.datasource.FileQueryScanNode.doFinalize(FileQueryScanNode.java:222) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.datasource.FileQueryScanNode.finalize(FileQueryScanNode.java:208) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.planner.PlanNode.finalize(PlanNode.java:722) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.planner.JoinNodeBase.finalize(JoinNodeBase.java:456) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.planner.PlanNode.finalize(PlanNode.java:722) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.planner.JoinNodeBase.finalize(JoinNodeBase.java:456) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.planner.PlanNode.finalize(PlanNode.java:722) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.planner.AggregationNode.finalize(AggregationNode.java:402) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.planner.PlanNode.finalize(PlanNode.java:722) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.planner.SetOperationNode.finalize(SetOperationNode.java:158) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.planner.PlanNode.finalize(PlanNode.java:722) ~[doris-fe.jar:1.2-SNAPSHOT]
```

This error msg is very confusing. So this PR return a more explicit
error msg.

### Release note

None
2025-05-08 10:31:37 +08:00
c637667394 branch-2.1:[fix](Nereids) when considering timezone to unix_timestamp, the boundary condition should changed (#49686) (#50669)
backport: https://github.com/apache/doris/pull/49686

Co-authored-by: LiBinfeng <libinfeng@selectdb.com>
2025-05-08 09:46:31 +08:00
9fe87a56bd [cherry-pick](branch-21) support lead/lag function input column as third params (#49381) (#50653)
### What problem does this PR solve?

cherry-pick from master (#49381)

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-05-07 23:08:56 +08:00
b9cfdcf46a branch-2.1: [chore](dependency)bump tomcat 8.5.86 to 9.0.104 #50622 (#50645)
Cherry-picked from #50622

Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>
2025-05-07 23:06:46 +08:00
22d846423d [fix](paimon)Set the target size of the split for 2.1 (#50596)
### What problem does this PR solve?

bp: #50083
2025-05-07 23:05:10 +08:00
6c62f28499 branch-2.1: [fix](RecycleBin) fix race condition issue when do execute ShowCatalogRecycleBinStmt (#35759) (#50654)
bp #35759

Co-authored-by: caiconghui <55968745+caiconghui@users.noreply.github.com>
Co-authored-by: caiconghui1 <caiconghui1@jd.com>
2025-05-07 23:02:16 +08:00
e52511ba1b branch-2.1: [chore](test) add ut for sql cache #49389 (#49549)
Cherry-picked from #49389

Co-authored-by: 924060929 <lanhuajian@selectdb.com>
2025-05-07 18:55:41 +08:00
e1c50c8e39 branch-2.1: [fix](mtmv)The change to the max_persistence_task_count configuration applies retroactively to existing MTMV #50537 (#50627)
Cherry-picked from #50537

Co-authored-by: zhangdong <zhangdong@selectdb.com>
2025-05-07 12:07:40 +08:00
a066374f94 [opt](paimon/iceberg)upgrade paimon to 1.0.1, iceberg to 1.6.1 for 2.1 (#50612)
bp #46990 #47768 #47117

Co-authored-by: Socrates <suyiteng@selectdb.com>
Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
2025-05-07 09:31:20 +08:00
1de9e7f9e8 branch-2.1: [fix](view)fix reset view def for restore wrong replace #50567 (#50600)
Cherry-picked from #50567

Co-authored-by: koarz <lihao@selectdb.com>
2025-05-06 20:15:45 +08:00
c3812b382b branch-2.1: [fix](nereids) fix bug of MergePercentileToArray #49773 (#50603)
Cherry-picked from #49773

Co-authored-by: feiniaofeiafei <moailing@selectdb.com>
2025-05-06 17:09:32 +08:00
a9b948f476 branch-2.1: [fix](catalog) do cache load when cache value is not present #50188 (#50451)
Cherry-picked from #50188

---------

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
Co-authored-by: morningman <yunyou@selectdb.com>
2025-05-06 10:59:11 +08:00
65151b46aa branch-2.1: [improve](routine load) add more metrics to observe the routine load job #48209 (#48765)
Cherry-picked from #48209

Co-authored-by: hui lai <laihui@selectdb.com>
2025-05-05 10:25:37 +08:00
98be2cedcf branch-2.1: [fix](constant fold)Make sure FE cast double to varchar generate identical result with BE. #50425 (#50548)
Cherry-picked from #50425

Co-authored-by: James <lijibing@selectdb.com>
2025-04-30 09:17:47 +08:00
2b62fb1a59 [fix](audit) fix potential audit log missing issue (#50357) (#50501)
bp #50357
2025-04-29 08:27:13 +08:00
fcc05d630b [chore](show engines)Change palo to Doris (#47861) (#50459)
bp #47861
2025-04-28 18:57:56 +08:00
d23bd87247 branch-2.1: [fix](txn) Restart fe after commit txn causes loadedTblIndexes lossing #50123 (#50206)
Cherry-picked from #50123

Co-authored-by: Uniqueyou <wangyixuan@selectdb.com>
2025-04-26 14:58:36 +08:00
90e22197ff [fix](json-functions)fix json-replace/insert/set/array behavior with complex type #50308 (#50309)
### What problem does this PR solve?
backport: https://github.com/apache/doris/pull/50308

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-04-24 19:04:25 +08:00
dd43f13549 branch-2.1:[fix](ut)Fix StmtExecutorTest (#50322) (#50362)
backport: https://github.com/apache/doris/pull/50322
2025-04-24 16:18:02 +08:00
91eed373be branch-2.1: [fix](array_map) fix array_map functions avoid core #50201 (#50331)
Cherry-picked from #50201

---------

Co-authored-by: amory <wangqiannan@selectdb.com>
2025-04-24 15:28:12 +08:00
cf72fa82e2 [Improve](explode) explode function support multi param (#50310)
### What problem does this PR solve?
backport:https://github.com/apache/doris/pull/48537
Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-04-23 23:27:07 +08:00
b0c8cb0818 branch-2.1: [Bug](function) fix Could not find function explode_json_array_json_outer #50164 (#50227)
Cherry-picked from #50164

Co-authored-by: zhangstar333 <zhangsida@selectdb.com>
2025-04-23 15:59:08 +08:00
98f5d95fba branch-2.1: [feat](function) SUBSTRING_INDEX function delimiter supports dynamic #50149 (#50303)
Cherry-picked from #50149

Co-authored-by: lw112 <131352377+felixwluo@users.noreply.github.com>
2025-04-23 15:00:40 +08:00
a193f16a5a branch-2.1:[fix](auditlog)Set sqlHash in executeInternalQuery (#49984) (#50271)
backport: https://github.com/apache/doris/pull/49984
2025-04-22 18:05:56 +08:00
1efe62c7ba branch-2.1: [opt](hive) add option to get schema from table object #50038 (#50269)
Cherry-picked from #50038

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
2025-04-22 14:25:03 +08:00
b9bb9e1611 branch-2.1: [fix](Export) fix the lock leak issue of Export #50088 (#50219)
Cherry-picked from #50088

Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
2025-04-22 14:21:42 +08:00
1c90fc6c02 branch-2.1: [fix](connect) fix wrong format causing connection fail #50214 (#50217)
bp #50214
2025-04-21 16:34:00 +08:00
188cbabb58 [enhance](auth)Remove restrictions on user creation and other operations when enabling ranger/LDAP (#50137)
### What problem does this PR solve?
- In version 2.1, the global permission check still calls the internal
permission interface. If grant is not allowed, it will be impossible to
assign admin and other permissions to users
- According to the current design of LDAP, if there is no user in LDAP,
Doris will check again to see if the user exists internally. If there
is, login will also be allowed. Therefore, creating users should not be
prohibited
2025-04-21 14:15:07 +08:00
d575a0921a [fix](hive) Fix hive metastore client socket timeout (#50192) 2025-04-21 14:09:20 +08:00
01b5cc4f1f [opt](mtmv) Doesn't throw npe when not set distribution info which should be random (#49402)
### What problem does this PR solve?

pr: https://github.com/apache/doris/pull/39427
commitId: 9ffb060a



Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-04-21 10:41:07 +08:00
481fca6766 [revert](statistics)Still use sample analyze sql when sample full table. (#50199)
Still use sample analyze sql when sample full table.
2025-04-21 10:12:01 +08:00
048537cd03 branch-2.1:[fix](nereids)Use utf-8 when convert string like literal to double. (#50085) (#50155)
backport: https://github.com/apache/doris/pull/50085
2025-04-19 17:16:20 +08:00
967d0a59b9 branch-2.1: [fix](load) reset routine load task EOF sign correctly #50048 (#50179)
Cherry-picked from #50048

Co-authored-by: hui lai <laihui@selectdb.com>
2025-04-19 17:15:18 +08:00
64815c6403 branch-2.1: [fix](dialect)fix starrocks dialect #50099 (#50165)
Cherry-picked from #50099

Co-authored-by: Jiwen liu <61498169+liujiwen-up@users.noreply.github.com>
2025-04-18 20:25:38 +08:00
4146cbbbbb branch-2.1-pick: [Opt](mow) Forbid time_series compaction policy on unique table (#49905) (#50132)
pick https://github.com/apache/doris/pull/49905
2025-04-18 11:34:18 +08:00
77ea907b54 branch-2.1: [Bug](materialized-view) Fixed the problem of using drop table force and create mv stmt at the… #41580 (#50133)
Cherry-picked from #41580

Co-authored-by: Pxl <pxl290@qq.com>
2025-04-18 11:30:47 +08:00
ea29bc523e branch-2.1: [Enhancement](GEO) Support Multipolygon and some spatial functions (#50073)
pick: https://github.com/apache/doris/pull/37003,
https://github.com/apache/doris/pull/48695 and
https://github.com/apache/doris/pull/49665

---------

Co-authored-by: Mryange <59914473+Mryange@users.noreply.github.com>
Co-authored-by: koi <koi20000@163.com>
2025-04-17 09:25:29 +08:00
2ef136cad6 [fix](variant) building index on the variant column is prohibited (#49159) (#50028)
pick from master #49159
2025-04-16 17:52:56 +08:00
bbdb057e2c branch-2.1: [fix](nereids) EliminateGroupByConstant should replace agg's group by after removing constant group by keys #49473 (#50044)
Cherry-picked from #49473

Co-authored-by: feiniaofeiafei <moailing@selectdb.com>
2025-04-16 13:02:39 +08:00