database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
21,979 Commits 1 Branch 0 Tags
cf3fe579ad5bd115d28cdf591728fa594549fafe
Commit Graph

313 Commits

Author SHA1 Message Date
zy-kkk
3a1e95c6c2 branch-2.1: [improvement](jdbc catalog) Optimize the acquisition of indentity type in SQLServer (#51659) 
pick #51285
 2025-06-16 16:50:37 +08:00
Socrates
dd89a78d30 branch-2.1:[fix](parquet/orc) Disable string dictionary filtering when predicate express is not binary pred and in pred (#50749) (#51266) 
cherry-pick: #50749
 2025-05-27 18:12:56 +08:00
github-actions[bot]
13fbc9efa6 branch-2.1: [fix](hive) fix write hive partition by Doris #50864 (#50921) 
Cherry-picked from #50864

Co-authored-by: Socrates <suxiaogang223@icloud.com>
 2025-05-17 16:14:23 +08:00
github-actions[bot]
0efd97055d branch-2.1: [fix](jdbc catalog) Improve conjunct expression handling in JdbcScanNode #50542 (#50648) 
Cherry-picked from #50542

Co-authored-by: zy-kkk <zhongyk10@gmail.com>
 2025-05-10 08:38:45 +08:00
github-actions[bot]
79056d4d7a branch-2.1: [feat](hive) add catalog level partition cache property #50724 (#50762) 
Cherry-picked from #50724

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
 2025-05-09 22:01:49 +08:00
github-actions[bot]
a9b948f476 branch-2.1: [fix](catalog) do cache load when cache value is not present #50188 (#50451) 
Cherry-picked from #50188

---------

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
Co-authored-by: morningman <yunyou@selectdb.com>
 2025-05-06 10:59:11 +08:00
Socrates
0710d9b2d6 branch-2.1: [fix](orc) Should not pass selection vector when decode child column of List or Map #50136 (#50316) 
bp: #50136
 2025-04-25 09:04:06 +08:00
github-actions[bot]
1efe62c7ba branch-2.1: [opt](hive) add option to get schema from table object #50038 (#50269) 
Cherry-picked from #50038

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
 2025-04-22 14:25:03 +08:00
github-actions[bot]
2aa9cb2276 branch-2.1: [fix](lzo) fix lzo decompression failed #49538 (#49634) 
Cherry-picked from #49538

---------

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
Co-authored-by: morningman <yunyou@selectdb.com>
 2025-04-17 09:19:35 +08:00
github-actions[bot]
06ab763efb branch-2.1: [fix](parquet)fix when hive_parquet_use_column_names=false && read partition tb cause be core. #49966 (#50008) 
Cherry-picked from #49966

Co-authored-by: daidai <changyuwei@selectdb.com>
 2025-04-16 12:44:48 +08:00
github-actions[bot]
0735c19cdd branch-2.1: [fix](paimon) Covert Paimon DeletionFile Path to StoragePath in fe #49645 (#49751) 
Cherry-picked from #49645

Co-authored-by: Socrates <suyiteng@selectdb.com>
 2025-04-02 20:25:10 +08:00
Socrates
94986fc574 branch-2.1: [fix](multi-catalog) Fix bug: "Can not create a Path from an empty string" (#49382) (#49641) 
### What problem does this PR solve?
Problem Summary:
In HiveMetaStoreCache, the function FileInputFormat.setInputPaths is
used to set input paths. However, this function splits paths using
commas, which is not the expected behavior. As a result, when partition
values contain commas, it leads to incorrect path parsing and potential
errors.
```java
  public static void setInputPaths(JobConf conf, String org.apache.hadoop.shaded.com.aSeparatedPaths) {
    setInputPaths(conf, StringUtils.stringToPath(
                        getPathStrings(org.apache.hadoop.shaded.com.aSeparatedPaths)));
  }
```
To prevent FileInputFormat.setInputPaths from splitting paths by commas,
we use another overloaded version of the method. Instead of passing a
comma-separated string, we explicitly pass a Path object, ensuring that
partition values containing commas are handled correctly.
```java
  public static void setInputPaths(JobConf conf, Path... inputPaths) {
    Path path = new Path(conf.getWorkingDirectory(), inputPaths[0]);
    StringBuffer str = new StringBuffer(StringUtils.escapeString(path.toString()));
    for(int i = 1; i < inputPaths.length;i++) {
      str.append(StringUtils.COMMA_STR);
      path = new Path(conf.getWorkingDirectory(), inputPaths[i]);
      str.append(StringUtils.escapeString(path.toString()));
    }
    conf.set(org.apache.hadoop.shaded.org.apache.hadoop.mapreduce.lib.input.
      FileInputFormat.INPUT_DIR, str.toString());
  }
```

### Release note

None
 2025-03-29 09:13:43 +08:00
Calvin Kirs
0f489dd7b7 [test](kerberos)Add hdfs(tvf),outfile and export test (#48433) (#48776) 
#48433
(cherry picked from commit 48fa09096d695c1f760b04ceb834789b1fde63dc)
 2025-03-07 14:29:48 +08:00
github-actions[bot]
661eba5c92 branch-2.1: [test](stable) fix some unstable hive and mc cases #48697 (#48770) 
Cherry-picked from #48697

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
 2025-03-06 20:25:01 +08:00
github-actions[bot]
bc6af178b3 branch-2.1: [fix](parquet)Fix data column and null map column not equal when reading Parquet complex type cross-page data #47734 (#48039) 
Cherry-picked from #47734

Co-authored-by: daidai <changyuwei@selectdb.com>
 2025-02-19 15:21:13 +08:00
zy-kkk
fb31586612 branch-2.1: [test](jdbc catalog) add more jdbc catalog extreme test (#47799) 
cherry-pick (#47525)
 2025-02-14 17:03:49 +08:00
zy-kkk
3ec900e878 branch-2.1:[opt](jdbc catalog) Compatible with higher ClickHouse JDBC Driver versions (#47832) 
cherry-pick from (#46026)
 2025-02-13 16:55:25 +08:00
Thearas
3aad9e5f67 [opt](oceanbase docker)Use LTS docker image and print unhealthy docker logs (#46647) (#47349) 
### What problem does this PR solve?

Problem Summary:
Oceanbase container sometimes start failed.
<img width="653" alt="image"

src="https://github.com/user-attachments/assets/d95c66cf-7e04-4179-a565-9b9dd8b87128"
/>

We do two things:
1. Print last 100 lines docker logs of unhealthy container for debugging
2. Upgrade Oceanbase docker image to the newest `4.2.1-lts`, since it is
7 months newer than `4.2.1`, more stable
 2025-01-24 11:22:02 +08:00
github-actions[bot]
6c72bfe082 branch-2.1: [fix](test) fix unstable test after some test migration #46834 (#46852) 
Cherry-picked from #46834

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
 2025-01-13 18:01:03 +08:00
daidai
94856b19a2 [fix](hive)fix select count(*) hive full acid tb opt error. (#46732) (#46804) 
bp #46732
 2025-01-12 08:55:10 +08:00
zy-kkk
04b840713b branch-2.1: [fix](jdbc catalog) Use factory methods to return mapped types instead (#46698) 
cherry-pick #46623
 2025-01-09 18:12:03 +08:00
github-actions[bot]
4d0037a928 branch-2.1: [fix](ES catalog)Fix query long value exception with doc_value #46554 (#46581) 
Cherry-picked from #46554

Co-authored-by: qiye <luen@selectdb.com>
 2025-01-08 15:26:58 +08:00
github-actions[bot]
012848060b branch-2.1: [fix](test) Fix mysql catalog check db test #46263 (#46343) 
Cherry-picked from #46263

Co-authored-by: zy-kkk <zhongyk10@gmail.com>
 2025-01-04 10:45:26 +08:00
github-actions[bot]
a6fc852ce7 branch-2.1: [opt](catalog) use table in db object return get db #46211 (#46229) 
Cherry-picked from #46211

Co-authored-by: zy-kkk <zhongyk10@gmail.com>
Co-authored-by: morningman <yunyou@selectdb.com>
 2025-01-01 08:00:49 +08:00
github-actions[bot]
6c9167d9d9 branch-2.1: [test](doc) add job-scheduler example in doris's doc to regression test #42904 (#45968) 
Cherry-picked from #42904

Co-authored-by: yagagagaga <zhangminke@selectdb.com>
 2024-12-30 10:44:58 +08:00
github-actions[bot]
5425e4686b branch-2.1: [fix](auth) fix use database stmt access unauthorized catalog #45720 (#45978) 
Cherry-picked from #45720

Co-authored-by: Petrichor <xiaowenli@selectdb.com>
 2024-12-30 10:43:28 +08:00
daidai
a380f5d222 [enchement](utf8)import enable_text_validate_utf8 session var (#45537) (#46070) 
bp #45537
 2024-12-28 10:05:03 +08:00
zy-kkk
02519c37ea branch-2.1: [test](mysql catalog) fix mysql jdbc catalog test case (#45988) 2024-12-26 15:06:24 +08:00
zy-kkk
bceebfcf38 branch-2.1: [test](jdbc catalog) add some jdbc query tvf test (#45991) 
cherry-pick #45938
 2024-12-26 15:05:50 +08:00
zy-kkk
1396f5a950 [2.1][improvement](jdbc catalog) Optimize JdbcCatalog case mapping stability (#43256) 
cherry-pick #41510
 2024-12-26 14:57:56 +08:00
James
289d621faa [improvement](information_schema)Show view definition in information_schema.views. (#45857) (#45930) 
backport: https://github.com/apache/doris/pull/45857
 2024-12-26 10:11:13 +08:00
daidai
303557ac70 [fix](hive)fix hive insert only translaction table. (#45753) 
### What problem does this PR solve?
bp #44001 , but no hive4 acid table.

Problem Summary:
1. Fixed the issue that when reading insert translaction only tables,
there was no acid check, which caused multiple data reads (i.e., reading
data from the previous base_n).
2. Forbidden to create, insert data, and delete aicd tables.
 2024-12-22 21:23:21 +08:00
Mingyu Chen (Rayner)
19c0e89da7 [enchement](iceberg)support read iceberg partition evolution table. (#45367) (#45569) 
cherry-pick #45367

Co-authored-by: daidai <changyuwei@selectdb.com>
 2024-12-20 08:56:51 +08:00
github-actions[bot]
d6c629d293 branch-2.1: [fix](tvf) Tvf supports to parse the enclose character in csv files #45407 (#45570) 
Cherry-picked from #45407

Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
 2024-12-19 16:23:44 +08:00
Socrates
7d32e4f71f branch-2.1: [Fix](ORC) Not push down fixed char type in orc reader #45484 (#45525) 
cherry-pick #45484
 2024-12-19 14:06:00 +08:00
Mingyu Chen (Rayner)
855e9a508c [fix](catalog) opt the count pushdown rule for iceberg/paimon/hive scan node (#44038) (#45564) 
bp #44038
 2024-12-18 09:54:56 +08:00
github-actions[bot]
b881157d52 branch-2.1: [regression](paimon) test Paimon's connectivity of cloud filesystems #45268 (#45378) 
Cherry-picked from #45268

Co-authored-by: Socrates <suyiteng@selectdb.com>
 2024-12-14 19:22:16 -08:00
wuwenchi
667f5e6e6a [feat](iceberg)Supports using rest type catalog to read tables in unity catalog for 2.1 (#43525) (#45217) 
bp: #43525
 2024-12-12 00:49:36 -08:00
Mingyu Chen (Rayner)
d4a6fd1850 Revert #43255 & #44615 (#45096) 
Revert "branch-2.1: [enhance](orc) Optimize ORC Predicate Pushdown for
OR-connected Predicate #43255 (#44438)"
Revert "[fix](orc) check all the cases before build_search_argument
(#44615) (#44801)"
 2024-12-06 21:14:13 +08:00
github-actions[bot]
dceaf97381 branch-2.1: [enhance](orc) Optimize ORC Predicate Pushdown for OR-connected Predicate #43255 (#44438) 
Cherry-picked from #43255

Co-authored-by: Socrates <suyiteng@selectdb.com>
 2024-11-22 22:52:53 +08:00
github-actions[bot]
ec068049c7 branch-2.1: [fix](meta-cache) fix refreshOnlyCatalogCache when use_meta_cache = false #44363 (#44461) 
Cherry-picked from #44363

Co-authored-by: Socrates <suyiteng@selectdb.com>
 2024-11-22 22:41:10 +08:00
daidai
702abbff0f [Opt](orc)Optimize the merge io when orc reader read multiple tiny stripes. (#42004) (#44239) 
bp #42004

Co-authored-by: kaka11chen <kaka11.chen@gmail.com>
 2024-11-22 11:01:41 +08:00
Mingyu Chen (Rayner)
fbf573f8c8 [fix](hive)fix hive catalog miss partition that have special characters. #42906 (#44152) 
cherry pick from #42906

---------

Co-authored-by: daidai <2017501503@qq.com>
 2024-11-21 14:11:22 +08:00
github-actions[bot]
83b74827aa branch-2.1: [fix](iceberg)Fix count(*) error with dangling delete problem #44039 (#44101) 
Cherry-picked from #44039

Co-authored-by: wuwenchi <wuwenchi@selectdb.com>
 2024-11-19 17:19:25 +08:00
github-actions[bot]
161d76328d branch-2.1: [fix](iceberg)Delete the data files when deleting the table #43441 (#44138) 
Cherry-picked from #43441

Co-authored-by: wuwenchi <wuwenchi@selectdb.com>
 2024-11-18 18:00:13 +08:00
github-actions[bot]
b120e895f1 branch-2.1: [fix](test) fix mysql jdbc catalog test out #43867 (#43916) 
Cherry-picked from #43867

Co-authored-by: zy-kkk <zhongyongkang@selectdb.com>
 2024-11-16 16:29:58 +08:00
github-actions[bot]
a1ff02288f branch-2.1: [fix](hive) support query hive view created by spark (#43553) 
Cherry-picked from #43530

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
Co-authored-by: morningman <yunyou@selectdb.com>
 2024-11-11 23:28:53 +08:00
github-actions[bot]
7de7541ec6 branch-2.1: [fix](ES Catalog)Only like on keyword can be applied to wildcard query (#43400) 
Cherry-picked from #41176

Co-authored-by: qiye <jianliang5669@gmail.com>
 2024-11-07 22:43:42 +08:00
Mingyu Chen (Rayner)
09f2516c7c [fix](test) fix some test cases #43217 (#43216) 
bp #43217
 2024-11-04 22:15:30 +08:00
Mingyu Chen (Rayner)
98d3db03b1 [fix](regression-test) fix test_hive_serde_prop #42886 (#43098) 
cherry pick from #42886

Co-authored-by: Socrates <suxiaogang223@icloud.com>
 2024-11-01 23:11:45 +08:00