Commit Graph

21831 Commits

Author SHA1 Message Date
87eeddc52c branch-2.1: [fix](regress) fix join_condition #50719 (#50738)
Cherry-picked from #50719

Co-authored-by: yujun <yujun@selectdb.com>
2025-05-09 22:02:45 +08:00
79056d4d7a branch-2.1: [feat](hive) add catalog level partition cache property #50724 (#50762)
Cherry-picked from #50724

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
2025-05-09 22:01:49 +08:00
f8de821d49 branch-2.1:[fix](regression)Fix test analyze mv case. (#50701) (#50751)
backport: https://github.com/apache/doris/pull/50701
2025-05-09 17:38:40 +08:00
0002500757 branch-2.1: [Fix](inverted index) fix rename column build index bug #50056 (#50732)
pick #47562 #50056 from master

---------

Co-authored-by: qiye <luen@selectdb.com>
2025-05-09 17:13:46 +08:00
fee5d40e07 [fix](planner) fix show variable display wrong enable_nereids_planner value (#50746)
### What problem does this PR solve?

fix show variable display wrong enable_nereids_planner value, introduced
by #49913

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [x] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [x] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-05-09 17:09:07 +08:00
04cbb2ac66 [fix](information_schema) fix backend_active_tasks table only return one backend's data (#50721) (#50722)
cherry pick from #50721
2025-05-09 15:01:22 +08:00
5923d97804 branch-2.1: [improvement](regression)Add log to print jdbc url in prepare stmt test #50711 (#50729)
Cherry-picked from #50711

Co-authored-by: James <lijibing@selectdb.com>
2025-05-09 14:37:45 +08:00
fde8d05f5d branch-2.1 [opt](nereids) catch all exceptions in StatsCalculator (#49415) (#50364) 2025-05-09 11:24:18 +08:00
9422c973af branch-2.1: [fix](binlog) Acquire migration lock before ingesting binlog #50663 (#50709)
cherry pick from #50663
2025-05-09 11:16:18 +08:00
0347e8a3c6 branch-2.1: proper planning of shadow columns for load and schema change concurrent execution (#49332) (#50710) 2025-05-09 11:15:08 +08:00
f0f0f21e5f [regression-test](case) move github_events to nonConcurrent (#50733) 2025-05-09 11:14:11 +08:00
37f3c8f0c7 [fix](nereids) fix fold constant return wrong scale of datetime type (#50142) (#50716)
cherry pick from #50142
2025-05-09 11:12:07 +08:00
bf5885a8f8 branch-2.1: [opt](Nereids) avoid generate nested alias expr when plan insert values (#50388) 2025-05-09 11:06:24 +08:00
df9d8f88cc [fix](case) fix some unstable p2 cases #50697 (#50702)
pick from master #50697
2025-05-09 09:23:08 +08:00
12e6a02158 branch-2.1: [enhance](mtmv)Exclude trigger table support db and catalog #49961 (#50554)
Cherry-picked from #49961

Co-authored-by: zhangdong <zhangdong@selectdb.com>
2025-05-09 09:22:31 +08:00
523681d58e branch-2.1: [enhance](mtmv)Only restrict MTMV to not allow concurrent insert overwrite execution #48673 (#49965)
Cherry-picked from #48673

Co-authored-by: zhangdong <zhangdong@selectdb.com>
2025-05-08 19:19:30 +08:00
1b108604d5 branch-2.1: [fix](function) fix error result in split_by_string with utf8 chars #40710 (#50689)
Cherry-picked from #40710

Co-authored-by: Mryange <59914473+Mryange@users.noreply.github.com>
2025-05-08 19:15:52 +08:00
ebe302cb7e branch-2.1: [fix](nereids) do eliminate constant group by key in normalizeagg #49589 (#50212)
Cherry-picked from https://github.com/apache/doris/pull/49589
2025-05-08 18:52:40 +08:00
995f1e5dc0 branch-2.1:[fix](Nereids) fix regression framework compare issue and fix code point count (#49575) (#50667)
backport: https://github.com/apache/doris/pull/49575

Co-authored-by: LiBinfeng <libinfeng@selectdb.com>
2025-05-08 16:53:02 +08:00
5501e130bf [fix](parquet)Fixed the problem that when Parquert reader use index to read files, there will be multiple threads modify same object (#50161) (#50496)
bp #50161
2025-05-08 15:51:25 +08:00
6f84c4d7e2 branch-2.1: [fix](planner) return explicit error msg when falling back to old planner (#50676)
### What problem does this PR solve?

Problem Summary:

When fall back to old planner, the partition pruning of external table
will fail,
and NPE will be throw like:
```
2025-04-14 16:22:33,427 WARN (mysql-nio-pool-19697|237161) [HiveScanNode.getSplits():189] get file split failed for table: dwd_log_fact_channel_track_result_hi
java.lang.NullPointerException: null
        at org.apache.doris.datasource.hive.source.HiveScanNode.getPartitions(HiveScanNode.java:137) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.datasource.hive.source.HiveScanNode.getSplits(HiveScanNode.java:171) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.datasource.FileQueryScanNode.createScanRangeLocations(FileQueryScanNode.java:366) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.datasource.FileQueryScanNode.doFinalize(FileQueryScanNode.java:222) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.datasource.FileQueryScanNode.finalize(FileQueryScanNode.java:208) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.planner.PlanNode.finalize(PlanNode.java:722) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.planner.JoinNodeBase.finalize(JoinNodeBase.java:456) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.planner.PlanNode.finalize(PlanNode.java:722) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.planner.JoinNodeBase.finalize(JoinNodeBase.java:456) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.planner.PlanNode.finalize(PlanNode.java:722) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.planner.AggregationNode.finalize(AggregationNode.java:402) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.planner.PlanNode.finalize(PlanNode.java:722) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.planner.SetOperationNode.finalize(SetOperationNode.java:158) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.planner.PlanNode.finalize(PlanNode.java:722) ~[doris-fe.jar:1.2-SNAPSHOT]
```

This error msg is very confusing. So this PR return a more explicit
error msg.

### Release note

None
2025-05-08 10:31:37 +08:00
c637667394 branch-2.1:[fix](Nereids) when considering timezone to unix_timestamp, the boundary condition should changed (#49686) (#50669)
backport: https://github.com/apache/doris/pull/49686

Co-authored-by: LiBinfeng <libinfeng@selectdb.com>
2025-05-08 09:46:31 +08:00
48778eab4d branch-2.1: [fix](iceberg)Fix the inconsistency between the data in pg and the data in MinIO. #50578 (#50641)
Cherry-picked from #50578

Co-authored-by: wuwenchi <wuwenchi@selectdb.com>
2025-05-07 23:15:02 +08:00
e64e5bc063 [Fix](orc-reader) Fixed issue with top level struct column having present stream failing to access repeatedly when late materialization occurs. (#50651)
### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note
Cherry-pick #50358

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-05-07 23:14:11 +08:00
9fe87a56bd [cherry-pick](branch-21) support lead/lag function input column as third params (#49381) (#50653)
### What problem does this PR solve?

cherry-pick from master (#49381)

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-05-07 23:08:56 +08:00
b9cfdcf46a branch-2.1: [chore](dependency)bump tomcat 8.5.86 to 9.0.104 #50622 (#50645)
Cherry-picked from #50622

Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>
2025-05-07 23:06:46 +08:00
22d846423d [fix](paimon)Set the target size of the split for 2.1 (#50596)
### What problem does this PR solve?

bp: #50083
2025-05-07 23:05:10 +08:00
3543124ed3 branch-2.1: [fix](systable) fix 'tables' systable's data_length filling error (#50595)
bp #50593
Also fix a bug that the checking logic for 'TTableStatus' is not right.
This bug only exists in branch-2.1
2025-05-07 23:04:46 +08:00
6c62f28499 branch-2.1: [fix](RecycleBin) fix race condition issue when do execute ShowCatalogRecycleBinStmt (#35759) (#50654)
bp #35759

Co-authored-by: caiconghui <55968745+caiconghui@users.noreply.github.com>
Co-authored-by: caiconghui1 <caiconghui1@jd.com>
2025-05-07 23:02:16 +08:00
ebcec779ec branch-2.1: [fix](function) fix error result when input utf8 in url_encode, strright, append_trailing_char_if_absent #49127 (#50660)
…ght, append_trailing_char_if_absent (#49127)

The url_encode function previously performed a modulus operation on a
signed number. Converting it to an unsigned number will fix the issue.
```
before
mysql> select url_encode('编码');
+----------------------+
| url_encode('编码')   |
+----------------------+
| %5.%23%0-%5.%10%/(   |
+----------------------+
now
mysql> select url_encode('编码');
+----------------------+
| url_encode('编码')   |
+----------------------+
| %E7%BC%96%E7%A0%81   |
+----------------------+
```

The strright function did not calculate the length according to the
number of UTF-8 characters.
```
before
mysql> select strright("你好世界",5);
+----------------------------+
| strright("你好世界",5)     |
+----------------------------+
|                            |
+----------------------------+
now

mysql> select strright("你好世界",5);
+----------------------------+
| strright("你好世界",5)     |
+----------------------------+
| 你好世界                   |
+----------------------------+
```

he case of inputting a UTF-8 character was not considered.
```
mysql> select append_trailing_char_if_absent('中文', '文');
+-------------------------------------------------+
| append_trailing_char_if_absent('中文', '文')    |
+-------------------------------------------------+
| NULL                                            |
+-------------------------------------------------+
now
mysql> select append_trailing_char_if_absent('中文', '文');
+-------------------------------------------------+
| append_trailing_char_if_absent('中文', '文')    |
+-------------------------------------------------+
| 中文                                            |
+-------------------------------------------------+
```
2025-05-07 22:37:50 +08:00
e52511ba1b branch-2.1: [chore](test) add ut for sql cache #49389 (#49549)
Cherry-picked from #49389

Co-authored-by: 924060929 <lanhuajian@selectdb.com>
2025-05-07 18:55:41 +08:00
e1c50c8e39 branch-2.1: [fix](mtmv)The change to the max_persistence_task_count configuration applies retroactively to existing MTMV #50537 (#50627)
Cherry-picked from #50537

Co-authored-by: zhangdong <zhangdong@selectdb.com>
2025-05-07 12:07:40 +08:00
a066374f94 [opt](paimon/iceberg)upgrade paimon to 1.0.1, iceberg to 1.6.1 for 2.1 (#50612)
bp #46990 #47768 #47117

Co-authored-by: Socrates <suyiteng@selectdb.com>
Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
2025-05-07 09:31:20 +08:00
02c3157e4c [branch-2.1](function) fix wrong floor of function date_diff when unit less than day (#49429) (#50606)
pick https://github.com/apache/doris/pull/49429
2025-05-07 09:27:37 +08:00
1de9e7f9e8 branch-2.1: [fix](view)fix reset view def for restore wrong replace #50567 (#50600)
Cherry-picked from #50567

Co-authored-by: koarz <lihao@selectdb.com>
2025-05-06 20:15:45 +08:00
babb98f10c [branch-2.1](function) fix wrong length check of function array_range (#49993) (#50605)
pick https://github.com/apache/doris/pull/49993
2025-05-06 18:57:03 +08:00
d667b5af38 [branch-2.1](function) Fix unexpected result of unix_timestamp when input is out of bound (#49430) (#50609)
pick https://github.com/apache/doris/pull/49430 but removed be-ut
because we can't.

Co-authored-by: Pxl <xl@selectdb.com>
2025-05-06 17:10:55 +08:00
c3812b382b branch-2.1: [fix](nereids) fix bug of MergePercentileToArray #49773 (#50603)
Cherry-picked from #49773

Co-authored-by: feiniaofeiafei <moailing@selectdb.com>
2025-05-06 17:09:32 +08:00
a9b948f476 branch-2.1: [fix](catalog) do cache load when cache value is not present #50188 (#50451)
Cherry-picked from #50188

---------

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
Co-authored-by: morningman <yunyou@selectdb.com>
2025-05-06 10:59:11 +08:00
65151b46aa branch-2.1: [improve](routine load) add more metrics to observe the routine load job #48209 (#48765)
Cherry-picked from #48209

Co-authored-by: hui lai <laihui@selectdb.com>
2025-05-05 10:25:37 +08:00
Pxl
6dee1fe623 [Chore](column) change ColumnFixedLengthObject log fatal to throw exception (#50540) 2025-05-03 23:04:01 +08:00
4aff17f355 branch-2.1: [fix](docker hive3) hive server oom and not auto-restart #50456 (#50507)
Cherry-picked from #50456

Co-authored-by: Thearas <gaozifeng@selectdb.com>
2025-05-03 22:44:29 +08:00
af4195e399 branch-2.1: [fix](geo) Fix ST_Contains behavior #50115 (#50569)
Cherry-picked from #50115

Co-authored-by: linrrarity <142187136+linrrzqqq@users.noreply.github.com>
2025-05-03 22:36:22 +08:00
98be2cedcf branch-2.1: [fix](constant fold)Make sure FE cast double to varchar generate identical result with BE. #50425 (#50548)
Cherry-picked from #50425

Co-authored-by: James <lijibing@selectdb.com>
2025-04-30 09:17:47 +08:00
dfc1dbfb59 branch-2.1: [bugfix](chdfs) fix hash collision in chdfs file system cache #49486 (#50545)
Cherry-picked from #49486

Co-authored-by: zhangyuan <ayuanzhang@tencent.com>
2025-04-30 09:16:37 +08:00
4b3dd6c10a branch-2.1: [feat](func) any function supports json #50311 (#50484)
Cherry-picked from #50311

Co-authored-by: lw112 <131352377+felixwluo@users.noreply.github.com>
2025-04-29 19:11:25 +08:00
5be2556503 [enhancement](compaction) replace hardcoded compaction thresholds with config (#49662) (#50514)
pick master #49662
2025-04-29 16:47:18 +08:00
3660139c64 branch-2.1: [fix](path gc) Fix path gc race with publish task #50343 (#50520)
cherry pick from #50343
2025-04-29 16:17:34 +08:00
4e8148105a [fix](serde)Fixed the issue that serde may cause be core when reading schema changed text table. (#50105) (#50504)
bp #50105
2025-04-28 21:54:43 -07:00
8cd9fd1e89 branch-2.1: [fix](memory) Fix thread_mem_tracker() performance (#50524) 2025-04-29 10:22:32 +08:00