Commit Graph

3705 Commits

Author SHA1 Message Date
79056d4d7a branch-2.1: [feat](hive) add catalog level partition cache property #50724 (#50762)
Cherry-picked from #50724

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
2025-05-09 22:01:49 +08:00
f0f0f21e5f [regression-test](case) move github_events to nonConcurrent (#50733) 2025-05-09 11:14:11 +08:00
df9d8f88cc [fix](case) fix some unstable p2 cases #50697 (#50702)
pick from master #50697
2025-05-09 09:23:08 +08:00
12e6a02158 branch-2.1: [enhance](mtmv)Exclude trigger table support db and catalog #49961 (#50554)
Cherry-picked from #49961

Co-authored-by: zhangdong <zhangdong@selectdb.com>
2025-05-09 09:22:31 +08:00
1b108604d5 branch-2.1: [fix](function) fix error result in split_by_string with utf8 chars #40710 (#50689)
Cherry-picked from #40710

Co-authored-by: Mryange <59914473+Mryange@users.noreply.github.com>
2025-05-08 19:15:52 +08:00
ebe302cb7e branch-2.1: [fix](nereids) do eliminate constant group by key in normalizeagg #49589 (#50212)
Cherry-picked from https://github.com/apache/doris/pull/49589
2025-05-08 18:52:40 +08:00
995f1e5dc0 branch-2.1:[fix](Nereids) fix regression framework compare issue and fix code point count (#49575) (#50667)
backport: https://github.com/apache/doris/pull/49575

Co-authored-by: LiBinfeng <libinfeng@selectdb.com>
2025-05-08 16:53:02 +08:00
9fe87a56bd [cherry-pick](branch-21) support lead/lag function input column as third params (#49381) (#50653)
### What problem does this PR solve?

cherry-pick from master (#49381)

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-05-07 23:08:56 +08:00
3543124ed3 branch-2.1: [fix](systable) fix 'tables' systable's data_length filling error (#50595)
bp #50593
Also fix a bug that the checking logic for 'TTableStatus' is not right.
This bug only exists in branch-2.1
2025-05-07 23:04:46 +08:00
02c3157e4c [branch-2.1](function) fix wrong floor of function date_diff when unit less than day (#49429) (#50606)
pick https://github.com/apache/doris/pull/49429
2025-05-07 09:27:37 +08:00
babb98f10c [branch-2.1](function) fix wrong length check of function array_range (#49993) (#50605)
pick https://github.com/apache/doris/pull/49993
2025-05-06 18:57:03 +08:00
d667b5af38 [branch-2.1](function) Fix unexpected result of unix_timestamp when input is out of bound (#49430) (#50609)
pick https://github.com/apache/doris/pull/49430 but removed be-ut
because we can't.

Co-authored-by: Pxl <xl@selectdb.com>
2025-05-06 17:10:55 +08:00
c3812b382b branch-2.1: [fix](nereids) fix bug of MergePercentileToArray #49773 (#50603)
Cherry-picked from #49773

Co-authored-by: feiniaofeiafei <moailing@selectdb.com>
2025-05-06 17:09:32 +08:00
a9b948f476 branch-2.1: [fix](catalog) do cache load when cache value is not present #50188 (#50451)
Cherry-picked from #50188

---------

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
Co-authored-by: morningman <yunyou@selectdb.com>
2025-05-06 10:59:11 +08:00
af4195e399 branch-2.1: [fix](geo) Fix ST_Contains behavior #50115 (#50569)
Cherry-picked from #50115

Co-authored-by: linrrarity <142187136+linrrzqqq@users.noreply.github.com>
2025-05-03 22:36:22 +08:00
4b3dd6c10a branch-2.1: [feat](func) any function supports json #50311 (#50484)
Cherry-picked from #50311

Co-authored-by: lw112 <131352377+felixwluo@users.noreply.github.com>
2025-04-29 19:11:25 +08:00
0710d9b2d6 branch-2.1: [fix](orc) Should not pass selection vector when decode child column of List or Map #50136 (#50316)
bp: #50136
2025-04-25 09:04:06 +08:00
90e22197ff [fix](json-functions)fix json-replace/insert/set/array behavior with complex type #50308 (#50309)
### What problem does this PR solve?
backport: https://github.com/apache/doris/pull/50308

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-04-24 19:04:25 +08:00
91eed373be branch-2.1: [fix](array_map) fix array_map functions avoid core #50201 (#50331)
Cherry-picked from #50201

---------

Co-authored-by: amory <wangqiannan@selectdb.com>
2025-04-24 15:28:12 +08:00
cf72fa82e2 [Improve](explode) explode function support multi param (#50310)
### What problem does this PR solve?
backport:https://github.com/apache/doris/pull/48537
Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-04-23 23:27:07 +08:00
b0c8cb0818 branch-2.1: [Bug](function) fix Could not find function explode_json_array_json_outer #50164 (#50227)
Cherry-picked from #50164

Co-authored-by: zhangstar333 <zhangsida@selectdb.com>
2025-04-23 15:59:08 +08:00
98f5d95fba branch-2.1: [feat](function) SUBSTRING_INDEX function delimiter supports dynamic #50149 (#50303)
Cherry-picked from #50149

Co-authored-by: lw112 <131352377+felixwluo@users.noreply.github.com>
2025-04-23 15:00:40 +08:00
1efe62c7ba branch-2.1: [opt](hive) add option to get schema from table object #50038 (#50269)
Cherry-picked from #50038

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
2025-04-22 14:25:03 +08:00
9123d73ed8 Branch-2.1 Cherry-pick [Fix](Variant) fix variant cast to jsonb into wrong NULL values (#50235)
cherry-pick from https://github.com/apache/doris/pull/50180
2025-04-22 09:23:38 +08:00
01b5cc4f1f [opt](mtmv) Doesn't throw npe when not set distribution info which should be random (#49402)
### What problem does this PR solve?

pr: https://github.com/apache/doris/pull/39427
commitId: 9ffb060a



Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-04-21 10:41:07 +08:00
c3a201dcbf branch-2.1: [fix](agg) prevent core dump in ColumnArray::size_at when handling type mismatch in streaming_agg_serialize_to_column #50001 (#50095)
### What problem does this PR solve?

Cherry-picked from: #50001
2025-04-17 09:30:06 +08:00
ea29bc523e branch-2.1: [Enhancement](GEO) Support Multipolygon and some spatial functions (#50073)
pick: https://github.com/apache/doris/pull/37003,
https://github.com/apache/doris/pull/48695 and
https://github.com/apache/doris/pull/49665

---------

Co-authored-by: Mryange <59914473+Mryange@users.noreply.github.com>
Co-authored-by: koi <koi20000@163.com>
2025-04-17 09:25:29 +08:00
2aa9cb2276 branch-2.1: [fix](lzo) fix lzo decompression failed #49538 (#49634)
Cherry-picked from #49538

---------

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
Co-authored-by: morningman <yunyou@selectdb.com>
2025-04-17 09:19:35 +08:00
fe634555bd [fix](variant)fix core in column_object when sort from empty block (#50035) 2025-04-16 14:03:04 +08:00
06ab763efb branch-2.1: [fix](parquet)fix when hive_parquet_use_column_names=false && read partition tb cause be core. #49966 (#50008)
Cherry-picked from #49966

Co-authored-by: daidai <changyuwei@selectdb.com>
2025-04-16 12:44:48 +08:00
b9d447741e Revert "branch-2.1: [Fix](Serde) Support hive compatible output format #49036" (#49986)
Reverts apache/doris#49831
2025-04-11 16:08:02 +08:00
9960b0960e branch-2.1: [bug](auto partition) Fix be crash with single replica insert (#48929)
Something may be wrong in #48536, and it reverts at #48926.

We need to re-run the pipeline and see what was wrong.
2025-04-11 15:23:51 +08:00
ba1ad12a21 branch-2.1: [Fix](Serde) Support hive compatible output format #49036 (#49831)
Cherry-picked from #49036

Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
2025-04-11 15:13:02 +08:00
308ede0657 branch-2.1-pick: [Fix](full compaction) Fix problems for full compaction #49825 (#49919)
pick https://github.com/apache/doris/pull/49825
2025-04-10 23:26:39 +08:00
8356141e03 branch-2.1: [enhancement](case) add cases for mow table load empty file #49843 (#49858)
Cherry-picked from #49843

Co-authored-by: MoanasDaddyXu <xujianxu@selectdb.com>
2025-04-08 14:04:30 +08:00
7c23af5b00 branch-2.1: [fix](opt)fix create expr tree to avoid the dangling reference #49082 (#49834)
Cherry-picked from #49082

Co-authored-by: zhangm365 <mzhang2048@gmail.com>
2025-04-08 12:01:50 +08:00
0ea87fba6f branch-2.1: [Bug][function] fix the string cast jsonb cause null map have not init value #49810 (#49817)
Cherry-picked from #49810

Co-authored-by: HappenLee <happenlee@selectdb.com>
2025-04-06 10:16:11 +08:00
c0bc16d88f [fix](function) wrong result of arrays_overlap (#49403) (#49707)
Pick #49403
If the two arrays have the same non-null elements, they are considered
overlapping, and the result is 1.
If the two arrays have no common non-null elements and either array
contains a null element, the result is null.
Otherwise, the result is 0.

```
select arrays_overlap([1, 2, 3], [1, null]);  -- result should be 1

select arrays_overlap([2, 3], [1, null]);  -- result should be null

select arrays_overlap([2, 3], [1]);   -- result should be 0
```

### What problem does this PR solve?
2025-04-04 20:58:01 +08:00
0735c19cdd branch-2.1: [fix](paimon) Covert Paimon DeletionFile Path to StoragePath in fe #49645 (#49751)
Cherry-picked from #49645

Co-authored-by: Socrates <suyiteng@selectdb.com>
2025-04-02 20:25:10 +08:00
e898dbbba0 branch-2.1: [fix](mc)Fixed the issue that maxcompute catalog can only read part of the timestamp data #49600 (#49706)
Cherry-picked from #49600

Co-authored-by: daidai <changyuwei@selectdb.com>
2025-04-01 17:09:15 +08:00
f2dac1f876 branch-2.1: [fix](test) fix unstable test infer_intersect_except #49000 (#49709) 2025-04-01 16:00:59 +08:00
cc0b2585ac branch-2.1: [function](date) Support date trunc function #49540 (#49661) 2025-03-29 20:21:33 +08:00
1259ee5088 branch-2.1: [Feature](function) support year of week #48870 (#49012) 2025-03-29 11:24:45 +08:00
94986fc574 branch-2.1: [fix](multi-catalog) Fix bug: "Can not create a Path from an empty string" (#49382) (#49641)
### What problem does this PR solve?
Problem Summary:
In HiveMetaStoreCache, the function FileInputFormat.setInputPaths is
used to set input paths. However, this function splits paths using
commas, which is not the expected behavior. As a result, when partition
values contain commas, it leads to incorrect path parsing and potential
errors.
```java
  public static void setInputPaths(JobConf conf, String org.apache.hadoop.shaded.com.aSeparatedPaths) {
    setInputPaths(conf, StringUtils.stringToPath(
                        getPathStrings(org.apache.hadoop.shaded.com.aSeparatedPaths)));
  }
```
To prevent FileInputFormat.setInputPaths from splitting paths by commas,
we use another overloaded version of the method. Instead of passing a
comma-separated string, we explicitly pass a Path object, ensuring that
partition values containing commas are handled correctly.
```java
  public static void setInputPaths(JobConf conf, Path... inputPaths) {
    Path path = new Path(conf.getWorkingDirectory(), inputPaths[0]);
    StringBuffer str = new StringBuffer(StringUtils.escapeString(path.toString()));
    for(int i = 1; i < inputPaths.length;i++) {
      str.append(StringUtils.COMMA_STR);
      path = new Path(conf.getWorkingDirectory(), inputPaths[i]);
      str.append(StringUtils.escapeString(path.toString()));
    }
    conf.set(org.apache.hadoop.shaded.org.apache.hadoop.mapreduce.lib.input.
      FileInputFormat.INPUT_DIR, str.toString());
  }
```

### Release note

None
2025-03-29 09:13:43 +08:00
4a31fc4e09 [Bug](fix) fix the percentile func result do not equal the percentile array rewrite result (#49379)
cherry pick https://github.com/apache/doris/pull/49351
2025-03-29 08:56:24 +08:00
89f4c90a44 [fix](named_struct) fix named_struct signature which deduce wrong for nested decimal precision (#49355) 2025-03-28 11:56:15 +08:00
aa43f84e72 [fix](case)rename case dir to prevent db name too long (#49580)
### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-03-27 17:02:48 +08:00
e2ab9ef46c [regression-test](cases) mv some cases nonConcurrent (#49460) 2025-03-26 11:22:44 +08:00
a40a4bbc67 branch-2.1: [fix](Nereids) fold constant for string function process emoji character by mistake #49087 (#49344)
pick: #49087 
Related PR: #40441

Problem Summary:

wrong calculation of emoji character length in some String function when
do constant folding in FE. For example:

select STRLEFT('😊😉👍', 2);

should return 😊😉, but fe return 😊 only when folding constant

fixed functions:
- left
- strleft
- right
- strright
- locate
- character_length
- split_by_string
- overlay
- replace_empty
2025-03-22 07:44:55 +08:00
c67bbe77ef [test](mtmv) Fix regression test not stable and add log for debug (#48483) (#49234) 2025-03-19 17:42:00 +08:00