database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
21,677 Commits 1 Branch 0 Tags
94986fc5746b0751daaf71d79af50dcd6004e39f
Commit Graph

8635 Commits

Author SHA1 Message Date
Socrates
94986fc574 branch-2.1: [fix](multi-catalog) Fix bug: "Can not create a Path from an empty string" (#49382) (#49641) 
### What problem does this PR solve?
Problem Summary:
In HiveMetaStoreCache, the function FileInputFormat.setInputPaths is
used to set input paths. However, this function splits paths using
commas, which is not the expected behavior. As a result, when partition
values contain commas, it leads to incorrect path parsing and potential
errors.
```java
  public static void setInputPaths(JobConf conf, String org.apache.hadoop.shaded.com.aSeparatedPaths) {
    setInputPaths(conf, StringUtils.stringToPath(
                        getPathStrings(org.apache.hadoop.shaded.com.aSeparatedPaths)));
  }
```
To prevent FileInputFormat.setInputPaths from splitting paths by commas,
we use another overloaded version of the method. Instead of passing a
comma-separated string, we explicitly pass a Path object, ensuring that
partition values containing commas are handled correctly.
```java
  public static void setInputPaths(JobConf conf, Path... inputPaths) {
    Path path = new Path(conf.getWorkingDirectory(), inputPaths[0]);
    StringBuffer str = new StringBuffer(StringUtils.escapeString(path.toString()));
    for(int i = 1; i < inputPaths.length;i++) {
      str.append(StringUtils.COMMA_STR);
      path = new Path(conf.getWorkingDirectory(), inputPaths[i]);
      str.append(StringUtils.escapeString(path.toString()));
    }
    conf.set(org.apache.hadoop.shaded.org.apache.hadoop.mapreduce.lib.input.
      FileInputFormat.INPUT_DIR, str.toString());
  }
```

### Release note

None
 2025-03-29 09:13:43 +08:00
github-actions[bot]
e8c91dcd73 branch-2.1: [fix](restore) correct the storage_medium of atomic restore #49330 (#49451) 
Cherry-picked from #49330

Co-authored-by: walter <maochuan@selectdb.com>
 2025-03-29 09:00:26 +08:00
walter
f55055096b branch-2.1: [improve](binlog) Allow commit txn without waiting txn publish #48961 (#49266) 
cherry pick from #48961
 2025-03-29 08:59:23 +08:00
github-actions[bot]
646f49fb93 branch-2.1: [fix](Nereids) use StringLikeLiteral as parameter type in constant folding #49413 (#49447) 
Cherry-picked from #49413

Co-authored-by: morrySnow <zhangwenxin@selectdb.com>
 2025-03-29 08:58:39 +08:00
HappenLee
4a31fc4e09 [Bug](fix) fix the percentile func result do not equal the percentile array rewrite result (#49379) 
cherry pick https://github.com/apache/doris/pull/49351
 2025-03-29 08:56:24 +08:00
github-actions[bot]
8f15e62de5 branch-2.1: [fix](fe) Using try-with-resource for auto close RemoteFileSystem #49637 (#49652) 
Cherry-picked from #49637

Co-authored-by: Lei Zhang <zhanglei@selectdb.com>
 2025-03-29 08:54:50 +08:00
amory
89f4c90a44 [fix](named_struct) fix named_struct signature which deduce wrong for nested decimal precision (#49355) 2025-03-28 11:56:15 +08:00
924060929
cf1938dd59 branch-2.1: [opt](nereids) skip run PruneOlapScanTablet when exists lots of InPredicate (#49387) 
cherry pick some parts from #47608 and #49386
 2025-03-28 11:51:20 +08:00
bobhan1
7d64a80959 branch-2.1-pick: [Fix](log) correct tablet diff log in TabletInvertedIndex.tabletReport (#49390) (#49445) 
pick https://github.com/apache/doris/pull/49390
 2025-03-28 11:50:31 +08:00
github-actions[bot]
57f04a7b9b branch-2.1: [fix](Nereids) fix double literal to string literal cast problem #49416 (#49523) 
Cherry-picked from #49416

Co-authored-by: LiBinfeng <libinfeng@selectdb.com>
 2025-03-28 11:49:02 +08:00
github-actions[bot]
92176c46bf branch-2.1: [feat](binlog) filter the async mv binlogs #49028 (#49099) 
Cherry-picked from #49028

Co-authored-by: walter <maochuan@selectdb.com>
 2025-03-28 10:01:00 +08:00
github-actions[bot]
2fb8e00907 branch-2.1: [chore](task) log the thrift message size if the broken pipe is occurred #49492 (#49509) 
Cherry-picked from #49492

Co-authored-by: walter <maochuan@selectdb.com>
 2025-03-28 09:57:53 +08:00
github-actions[bot]
b16821f018 branch-2.1: [fix](auth)Prohibit other users from modifying the root #48752 (#49585) 
Cherry-picked from #48752

Co-authored-by: zhangdong <zhangdong@selectdb.com>
 2025-03-28 09:54:32 +08:00
xy720
1ff0f70460 branch-2.1: [case](restore) Rename repo name of backup-restore regression cases with suite name (#49535) 
### What problem does this PR solve?

pick:https://github.com/apache/doris/pull/49237

1、Rename the repo of backup restore regression cases with a suite name
prefix.
2、Change the maximum length of repo names from 63 to 255

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [x] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [x] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
 2025-03-27 20:13:30 +08:00
yujun
ed56e84b31 branch-2.1: [fix](system) fix alter system modify hostname with valid ip exception #49551 (#49561) 
cherry pick from #49551
 2025-03-27 19:49:27 +08:00
minghong
374b901717 branch-2.1 [fix](nereids)avoid generate Runtime filter whose target is not a base table column #48804 (#49314) 2025-03-27 19:48:37 +08:00
github-actions[bot]
0a6bb54e47 branch-2.1: [opt](metrics) optimize performance of metrics endpoint #49380 (#49455) 
Cherry-picked from #49380

Co-authored-by: 924060929 <lanhuajian@selectdb.com>
 2025-03-26 11:31:06 +08:00
James
1fb771d526 branch-2.1: [fix](statistics)Fix replace table doesn't remove table stat meta memory leak bug. (#49345) (#49367) 
Cherry-picked from https://github.com/apache/doris/pull/49345
 2025-03-23 09:55:47 +08:00
Mingyu Chen (Rayner)
c3fad29a4f [fix](webui) add connection context to avoid NPE (#49213) (#49353) 
bp #49213
 2025-03-22 08:00:49 +08:00
LiBinfeng
a40a4bbc67 branch-2.1: [fix](Nereids) fold constant for string function process emoji character by mistake #49087 (#49344) 
pick: #49087 
Related PR: #40441

Problem Summary:

wrong calculation of emoji character length in some String function when
do constant folding in FE. For example:

select STRLEFT('😊😉👍', 2);

should return 😊😉, but fe return 😊 only when folding constant

fixed functions:
- left
- strleft
- right
- strright
- locate
- character_length
- split_by_string
- overlay
- replace_empty
 2025-03-22 07:44:55 +08:00
github-actions[bot]
2eed4f3a65 branch-2.1: [opt](paimon)Add suppressed information display #48947 (#48997) 
Cherry-picked from #48947

Co-authored-by: wuwenchi <wuwenchi@selectdb.com>
 2025-03-22 07:42:45 +08:00
github-actions[bot]
abc535a9e7 branch-2.1: [fix](hudi) Fix Memory Leak in BitCaskDiskMap Due to Circular Reference #48955 (#49115) 
Cherry-picked from #48955

Co-authored-by: Socrates <suyiteng@selectdb.com>
 2025-03-21 22:44:35 +08:00
qiye
71824569a9 [fix](Index)Make column unique ids in index dynamically computed (#48988) (#49300) 
bp #48988
 2025-03-21 19:51:00 +08:00
James
7b28e33815 [improvement](statistics)Remove read lock when doing db analyze. (#49250) (#49320) 
backport: https://github.com/apache/doris/pull/49250
 2025-03-21 18:04:38 +08:00
LiBinfeng
b130500203 [fix](Nereids) initcap constant folding should upper first character in all words (#49061) (#49342) 2025-03-21 18:03:28 +08:00
zhangdong
54b3000de5 [fix](auth)create view check select_priv of table instead of column (#49268) 2025-03-20 23:08:58 +08:00
seawinde
17ffd301fb [fix](mtmv) Fix collecting mv candidates when dml controlled by enable_dml_materialized_view_rewrite switch #48374 (#49263) 
### What problem does this PR solve?
pr: https://github.com/apache/doris/pull/48374
commitId: 0c9ce720
 2025-03-20 10:01:04 +08:00
seawinde
c67bbe77ef [test](mtmv) Fix regression test not stable and add log for debug (#48483) (#49234) 2025-03-19 17:42:00 +08:00
James
cc41d61eec [fix](nereids)Fix dlog1, trim, extract_url_parameter and parse_url FE constant calculate bug. (#49074) (#49224) 
backport: https://github.com/apache/doris/pull/49074
 2025-03-19 10:05:01 +08:00
github-actions[bot]
a32a7ba5eb branch-2.1: [fix](Nereids) deep copy for LogicalWindow is wrong #48861 (#49014) 
Cherry-picked from #48861

Co-authored-by: morrySnow <zhangwenxin@selectdb.com>
 2025-03-19 09:57:15 +08:00
github-actions[bot]
13c174df4b branch-2.1: [fix](Nereids) fix substring with only one parameter #48957 (#49030) 
Cherry-picked from #48957

Co-authored-by: LiBinfeng <libinfeng@selectdb.com>
 2025-03-19 09:56:13 +08:00
github-actions[bot]
44b868de6b branch-2.1: [fix](nereids) fix to_monday('1970-01-04 23:59:59') #49153 (#49176) 
Cherry-picked from #49153

Co-authored-by: yujun <yujun@selectdb.com>
 2025-03-18 18:07:26 +08:00
LiBinfeng
bd5d0ca741 branch-2.1: [fix](Nereids) fold str_to_date to wrong result when parameter out of range #49033 (#49150) 2025-03-18 11:01:49 +08:00
github-actions[bot]
68f50b569b branch-2.1: [fix](group commit)Fix replay wal fail problem on agg state type #49081 (#49143) 
Cherry-picked from #49081

Co-authored-by: huanghaibin <huanghaibin@selectdb.com>
 2025-03-18 10:57:20 +08:00
github-actions[bot]
98a59f914a branch-2.1: [chore](binlog) GetMeta returns dropped partition/table/index commit seq #48852 (#48900) 
Cherry-picked from #48852

Co-authored-by: walter <maochuan@selectdb.com>
 2025-03-17 14:03:09 +08:00
github-actions[bot]
cf88db4938 branch-2.1: [improvement](ccr) Add and adjust result for get_lag #48953 (#49055) 
Cherry-picked from #48953

Co-authored-by: Uniqueyou <wangyixuan@selectdb.com>
 2025-03-17 10:50:07 +08:00
github-actions[bot]
e7d4dda3c8 branch-2.1: [Fix](function) Fix wrong FE fold constant implementation of function date_format #49032 (#49086) 
Cherry-picked from #49032

Co-authored-by: zclllyybb <zhaochangle@selectdb.com>
 2025-03-16 20:16:40 +08:00
github-actions[bot]
be3e2f36d0 branch-2.1: [fix](nereids)fix the cascadesContext.getMemo()==null #48771 (#49095) 
Cherry-picked from #48771

Co-authored-by: zhangm365 <mzhang2048@gmail.com>
 2025-03-15 12:20:02 +08:00
yujun
1eff4538aa branch-2.1: [fix](nereids) fix convert to date literal throw exception #48980 (#48985) 2025-03-15 10:44:40 +08:00
James
3b70606c4f [improvement](statistics)Improve analyze partition column and key column corner case. (#48757) (#49101) 
backport: https://github.com/apache/doris/pull/48757
 2025-03-15 10:23:36 +08:00
924060929
8589db3ec3 Revert "branch-2.1: [fix](nereids) fix distinct window compute wrong result (#48987) (#49010)" (#49079) 
Revert "branch-2.1: [fix](nereids) fix distinct window compute wrong
result (#48987) (#49010)"
 2025-03-15 00:02:10 +08:00
github-actions[bot]
d0f6edd212 branch-2.1: [improvement](statistics)Add session variable for partition sample count. #48218 (#49092) 
Cherry-picked from #48218

Co-authored-by: James <lijibing@selectdb.com>
 2025-03-14 20:43:00 +08:00
James
ed3a979904 [feature](statistics)Support enable/disable auto analyze for table. (#39532) (#48863) 
backport: https://github.com/apache/doris/pull/39532
 2025-03-14 17:43:13 +08:00
github-actions[bot]
b5c85e0962 branch-2.1: [fix](hudi) replace non thread safe SimpleDateFormat #48923 (#49022) 
Cherry-picked from #48923

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
 2025-03-14 17:31:04 +08:00
James
5988495576 [fix](statistics)Control memory use for sample partition column and key column. (#46534) (#48927) 
backport: https://github.com/apache/doris/pull/46534
 2025-03-14 17:18:19 +08:00
yujun
2febf0de1b branch-2.1: [fix](udf) alias udf skip check enable_java_udf #48843 (#48914) 2025-03-14 11:21:01 +08:00
924060929
7123bbfecb branch-2.1: [fix](nereids) fix distinct window compute wrong result (#48987) (#49010) 
cherry pick from #48987
 2025-03-14 10:38:16 +08:00
github-actions[bot]
6d2abf9df3 branch-2.1: [Improvement]Add query start datetime #48906 (#48992) 
Cherry-picked from #48906

Co-authored-by: wangbo <wangbo@selectdb.com>
 2025-03-13 17:40:25 +08:00
924060929
ea59465ec7 branch-2.1: [fix](sql cache) fix prepare statement with sql cache throw NullPointerException (#48902) (#48977) 
cherry pick from #48902
 2025-03-13 14:35:58 +08:00
Mingyu Chen (Rayner)
fe37867846 Revert "branch-2.1:[fix](hudi) Fix Memory Leak in BitCaskDiskMap Due to Circular Reference" (#48974) 
Reverts apache/doris#48916
This is under verification, revert it now
 2025-03-12 19:07:49 +08:00