Commit Graph

21993 Commits

Author SHA1 Message Date
abb9c88a3f branch-2.1: [improve](hive row count)Log stack info when fetch hive row count fail. (#52795) (#52849)
backport: https://github.com/apache/doris/pull/52795
2025-07-09 09:57:24 +08:00
3dde6775a9 [Fix](case) Fix wg regression test (#52895) 2025-07-09 09:56:43 +08:00
29fe672dbc branch-2.1: [fix](ui) fix ui builds failed error #52711 (#52776)
Cherry-picked from #52711

Co-authored-by: Jeffrey <color.dove@gmail.com>
2025-07-05 09:03:09 +08:00
b2e70e938c branch-2.1: [improve](statistics)Specify column name when insert into stat table.(#52441) (#52633)
backport: https://github.com/apache/doris/pull/52441
2025-07-04 14:21:01 +08:00
0f7f7dd0f6 [Opt](multi-catalog)Disable dict filter in parquet/orc reader if have non-single conjuncts. (#52617)
### What problem does this PR solve?

Problem Summary:

Cherry-pick #44777
2025-07-04 12:21:21 +08:00
820971a384 branch-2.1: [regression-test](Variant) add escaped chars cases #52657 (#52699)
Cherry-picked from #52657

Co-authored-by: lihangyu <lihangyu@selectdb.com>
2025-07-04 11:18:26 +08:00
721f5965cf [fix](inverted index) fix memory leaks in inverted index (#52747) 2025-07-04 10:55:47 +08:00
8c850274bb [Bug](topn) variant column read in topn may coredump (#52583)
### What problem does this PR solve?

cherry pick #52573
Problem Summary:
2025-07-03 17:59:12 +08:00
b0e83ca7d1 branch-2.1: [fix](fe) multi statements different behavior in master and follower (#52144) (#52632)
pick https://github.com/apache/doris/pull/52144
2025-07-03 14:53:19 +08:00
6404277795 [fix](json) Add . after in JSON path to support correct token parsing (#52543) (#52544)
Boost tokenizer requires explicit "." after "$" to correctly extract
JSON path tokens. Without this, expressions like "$[0].key" cannot be
properly split, causing issues in downstream logic. This commit ensures
a "." is automatically added after "$" to maintain consistent token
parsing behavior.
2025-07-03 14:36:53 +08:00
fb70742e87 branch-2.1: [Fix](field) Fix potential memory leak and wrong binary reading about JsonbField (#50174) (#52693)
pick https://github.com/apache/doris/pull/50174
2025-07-03 12:38:37 +08:00
3c88f4df70 [opt](nereids) Improve sql parse performance by avoid parse twice when collect hint map (#52627)
cherry pick part of code from
pr: https://github.com/apache/doris/pull/40202
commitId: 81f3c484 


### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-07-03 08:56:16 +08:00
d374b6e1e4 [fix](replica) skip missing version should care catchup (#49999) (#50094) (#52386)
pick  (#49999) (#50094)
2025-07-03 08:32:06 +08:00
cabd664c7a [fix](case) Fix case test_index_like_select (#52674)
1. Numeric types can use LIKE
2025-07-03 08:29:03 +08:00
cf3fe579ad [fix](regression-test) fix variant_p2 case to show load failure #52652 (#52686) 2025-07-03 08:27:35 +08:00
ea8ef605a6 [fix](main) check error immediately after flight_server init (#52616) (#52624)
backport #52616
2025-07-02 10:21:38 +08:00
06bbfd1eea branch-2.1: [fix](nereids) fix tanh cosntant folding #52100 (#52527)
Cherry-picked from #52100

Co-authored-by: admiring_xm <86338598+wumeibanfa@users.noreply.github.com>
2025-07-02 10:21:10 +08:00
794982a668 branch-2.1: [fix](type) Fix TYPE_DATETIMEV2 upgrade and downgrade compatibility (#52530)
Introduced from https://github.com/apache/doris/pull/49643
2025-07-02 09:24:48 +08:00
577ef510d8 branch-2.1: [fix](mow) fix potential mem leak for DeleteBitmap::AggCache #52596 (#52613)
Cherry-picked from #52596

Co-authored-by: zhannngchen <zhangchen@selectdb.com>
2025-07-02 09:20:30 +08:00
dc8102bbf4 branch-2.1: [Bug](distinct) fix distinct function with over return error result #51875 (#52313)
Cherry-picked from #51875

Co-authored-by: zhangstar333 <zhangsida@selectdb.com>
2025-06-30 09:45:50 +08:00
fecc6ac802 branch-2.1: [fix](nereids) exceed expr limit error not fallback to legacy planner (#52431)
### What problem does this PR solve?

Legacy planner check expr limit is not reliable. When planner expr
change, it need explict call expr.analyse to check the limit.

Nereids planner check expr limit at expr's construct function, it's
reliable.

So nereids don't fallback to legacy planner when met exceed expr limit
error, this can avoid legacy planner forget to check the expr limit and
use a lot of memory.


### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-06-29 08:34:50 +08:00
ed94a6d21a branch-2.1: [fix](docker)Add docker-ps 'sudo' permissions #52395 (#52457)
Cherry-picked from #52395

Co-authored-by: wuwenchi <wuwenchi@selectdb.com>
Co-authored-by: wuwenchi.wwc <wuwenchi.wwc@oceanbase.com>
2025-06-28 20:06:51 +08:00
8480691871 branch-2.1: [fix](hive) ignore all hidden dir and files #52286 (#52324)
Cherry-picked from #52286

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
2025-06-28 14:23:27 +08:00
e923acef1b branch-2.1: [Fix](JsonReader) Fix the issue where the null bitmap of the JSON reader was not initialized when the JSON path is specified as '$.’ #52211 (#52268)
Cherry-picked from #52211

Co-authored-by: lihangyu <lihangyu@selectdb.com>
2025-06-28 14:21:38 +08:00
4999816ae6 branch-2.1: [fix](map) fix createMap function #52231 (#52401)
Cherry-picked from #52231

Co-authored-by: amory <wangqiannan@selectdb.com>
2025-06-28 14:19:48 +08:00
93e5431bb9 branch-2.1: [test](mv) Fix mv regression test is not chosen stable by CBO #50690 (#52353)
Cherry-picked #50690
2025-06-28 14:19:14 +08:00
1deb221f8c branch-2.1: [chore](ci)Replace local setup-maven with setup-java for more stable and simplified Maven setup #52229 (#52257)
Cherry-picked from #52229

Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>
2025-06-28 11:13:19 +08:00
cb60589c0a branch-2.1: [fix](trash) Fix shadow variable causing garbage scheduling time errors #51647 (#51996)
Cherry-picked from #51647

Co-authored-by: deardeng <dengxin@selectdb.com>
2025-06-28 11:12:25 +08:00
c671a09a56 branch-2.1: [fix](iceberg)Ensure proper authentication context before accessing Iceberg Catalog #52149 (#52234)
Cherry-picked from #52149

Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>
Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
2025-06-28 10:59:03 +08:00
95f10aa8a1 branch-2.1:[fix](mtmv) fix when compatible fail, will throw NPE (#49875) (#52273)
pick: https://github.com/apache/doris/pull/49875
2025-06-28 10:54:44 +08:00
af070ac853 [branch-2.1](timezone) Fix incorrect DST handling (#51454) (#52418)
pick https://github.com/apache/doris/pull/51454
2025-06-28 10:48:58 +08:00
e029e420ae branch-2.1: [fix](nereids) fix calc repeat property stack overflow #52159 (#52188)
cherry pick from #52159
2025-06-28 10:22:48 +08:00
5d20a04b32 branch-2.1: [fix](auth)Only treat admin@% and root@% as system users. #50904 (#52283)
Cherry-picked from #50904

Co-authored-by: zhangdong <zhangdong@selectdb.com>
2025-06-28 10:11:13 +08:00
86ea0f2b81 [fix](be) Fix be aws-sdk-cpp memory leak (#52185) (#52298)
* https://github.com/aws/aws-sdk-cpp/pull/2314
* In the aws-sdk-cpp code, there is a suggestion: "Please call
ShutdownAPI from the same thread from which InitAPI() has been called"
otherwise it will cause memory leak.

```
=================================================================
==3271527==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 216 byte(s) in 1 object(s) allocated from:
    #0 0x555f9ecf55ee in malloc (/mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be+0x370815ee) (BuildId: 22b637395dd039b0)
    #1 0x555fd8e19bd7 in CRYPTO_zalloc (/mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be+0x711a5bd7) (BuildId: 22b637395dd039b0)
    #2 0x555fda7c9d14 in aws_mqtt_library_init (/mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be+0x72b55d14) (BuildId: 22b637395dd039b0)
    #3 0x555fda74ed1f in Aws::Crt::ApiHandle::ApiHandle(aws_allocator*) (/mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be+0x72adad1f) (BuildId: 22b637395dd039b0)
    #4 0x555fa365f0c9 in doris::S3ClientFactory::instance() /root/doris/be/src/util/s3_util.cpp:188:28
    #5 0x555f9f13b87b in doris::io::ObjClientHolder::init() /root/doris/be/src/io/fs/s3_file_system.cpp:78:15
    #6 0x555f9f140eee in doris::io::S3FileSystem::init() /root/doris/be/src/io/fs/s3_file_system.cpp:182:21
    ......
```

### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-06-28 10:10:32 +08:00
d7184af32b branch-2.1: [fix](cluster)fix fe host may be contains Scope Identifier #52076 (#52398)
Cherry-picked from #52076

Co-authored-by: zhangdong <zhangdong@selectdb.com>
2025-06-28 10:09:22 +08:00
74a409d74e [test](mv)forbid create mv test case (#52350)
pick from #51777 
[test](mv)forbid create mv test case
2025-06-28 10:08:19 +08:00
307c047377 branch-2.1: [fix](Prepared Statment) Fix exec prepared insert stmt in non master error (#48689) (#52265)
backport: https://github.com/apache/doris/pull/48689

Co-authored-by: Lijia Liu <liutang123@yeah.net>
2025-06-28 09:57:53 +08:00
f0ce208e68 [fix](test) Create mv but not check finished (#52167) 2025-06-28 09:48:44 +08:00
16777c7235 branch-2.1: [fix](nereids) correct the log class in the command (#43896) (#52253)
backport: https://github.com/apache/doris/pull/43896

Co-authored-by: Yao-MR <yao.mr.cn@gmail.com>
2025-06-25 14:51:30 +08:00
43b99ddd7a [fix](mtmv) Fix compensate union all wrongly when partition mv use partition external table (#52218)
Fix compensate union all wrongly when partition mv use partition
external table

in test mv/external_table/part_partition_invalid.groovy
order_qt_after_modify_data_and_refresh_catalog data should be as
following:
```
1	2	1	2023-10-17	2023-10-17
2	2	2	2023-10-18	2023-10-18
3	2	3	2023-10-19	2023-10-19
```


### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #49514 

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [x] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-06-25 14:50:47 +08:00
6dca968e96 branch-2.1:[fix](planner)Fix the concurrency issue caused by Expr setting member… (#52194)
… variables (#51389)

pick: https://github.com/apache/doris/pull/51389
2025-06-25 11:45:27 +08:00
b3588b01c0 branch-2.1: [improvement](nereids)Support GROUP BY ... WITH ROLLUP syntax #51948 (#51978)
Cherry-picked from #51948

Co-authored-by: James <lijibing@selectdb.com>
2025-06-25 09:59:02 +08:00
042e18068a [chore](test) fix regression-test/suites/fault_injection_p0/test_variant_bloom_filter.groovy (#52201) 2025-06-24 18:23:49 +08:00
93d7eb57bc branch-2.1: [fix](hudi catalog) Fix the Kerberos authentication error when querying hudi table #51713 (#51903)
Cherry-picked from #51713

Co-authored-by: heguanhui <hgh_wy163mail@163.com>
2025-06-24 14:20:47 +08:00
667510f71b [branch-2.1]Remove memory sum check (#52055) 2025-06-24 14:08:28 +08:00
4cb66f3a40 branch-2.1: [chore](http) return NOT_FOUND if dir is not exists #51897 (#52156)
Cherry-picked from #51897

Co-authored-by: walter <maochuan@selectdb.com>
2025-06-24 10:39:28 +08:00
490963634b [enhancement](time_series) increase the version limit for the time series table (#51371) (#52077)
pick master #51371
2025-06-24 09:44:24 +08:00
c9e2df607b [opt](metrics) add metrics pipeline_task_queue_size (#51878) (#52141)
### What problem does this PR solve?

pick #51878 to branch-2.1
2025-06-24 09:27:13 +08:00
347c1e1e63 [branch-2.1]refactor workload group cpu hard limit (#51278) (#51382)
pick #51278
2025-06-23 14:45:45 +08:00
d194014914 [fix](ip) fix ip type in IndexDef support (#50637)
### What problem does this PR solve?
only in branch-2.1  we do not support old indexDef to support Ip type 
and fix array_contains && arrays_overlap for with type array<ip> which
backport: https://github.com/apache/doris/pull/50637
Issue Number: close #xxx
2025-06-20 19:51:38 +08:00