database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
19,529 Commits 1 Branch 0 Tags
54787f51c7ed454feff1663c4e4255c011749421
Commit Graph

152 Commits

Author SHA1 Message Date
苏小刚
f7068b5658 [cherry-pick](branch-2.1) Make doris read hive text table parameters and behavior consistent with hive (#37840) 
## Proposed changes

pick from master https://github.com/apache/doris/pull/37638

<!--Describe your changes.-->
 2024-07-16 22:24:50 +08:00
qiye
e5339a4014 [feature](ES Catalog)Support control scroll level by config #37180 (#37290) 
## Proposed changes

backport #37180
 2024-07-15 16:41:38 +08:00
slothever
16de141743 [regression](kerberos)add hive kerberos docker regression env (#37657) 
## Proposed changes
pick:
[regression](kerberos)fix regression pipeline env when write hosts 
(#37057)
[regression](kerberos)add hive kerberos docker regression env (#36430)
 2024-07-15 09:35:39 +08:00
Mingyu Chen
56a207c3f0 [case](paimon/iceberg)move cases from p2 to p0 (#37276) (#37738) 
bp #37276

Co-authored-by: wuwenchi <wuwenchihdu@hotmail.com>
 2024-07-13 10:01:05 +08:00
Mingyu Chen
81360cf897 [opt](test) shorten the external p0 running time (#37320) (#37473) 
bp #37320
 2024-07-09 15:35:15 +08:00
qiye
f8cee439b6 [feature](ES Catalog) map nested/object type in ES to JSON type in Doris (#37101) (#37182) 
backport #37101
 2024-07-05 10:48:32 +08:00
Mingyu Chen
55636e8035 [test](migrate) move 3 cases from p2 to p0 (#36957) (#37264) 
bp #36957

Co-authored-by: slothever <18522955+wsjz@users.noreply.github.com>
 2024-07-04 20:09:59 +08:00
Mingyu Chen
3613413a54 [fix](hive) support find serde info from both tbl properties and serde properties (#37043) (#37188) 
bp #37043
 2024-07-04 13:55:38 +08:00
Jibing-Li
bf3ea1839c [test]Mv external p2 test case to p0. (#37070) (#37140) 
backport: https://github.com/apache/doris/pull/37070
 2024-07-04 11:19:31 +08:00
zy-kkk
a9f9113c48 [branch-2.1][test](external)move hive cases from p2 to p0 (#37149) 
pk (#36855)
test_hive_same_db_table_name
test_hive_special_char_partition
test_complex_types
test_wide_table
 2024-07-03 19:44:52 +08:00
Mingyu Chen
e5695e058f [test](migrate) move 2 cases from p2 to p0 (#36935) (#37200) 
bp #36935

Co-authored-by: zhangdong <493738387@qq.com>
 2024-07-03 17:29:01 +08:00
Qi Chen
e857680661 [Migrate-Test](multi-catalog) Migrate p2 tests from p2 to p0. (#37175) 
Backport #36989.
 2024-07-03 11:08:49 +08:00
wuwenchi
e7e1e967cf [test](migrate) move 2 cases from p2 to p0 for 2.1 (#37139) 
pick #37004
 2024-07-02 22:50:53 +08:00
Tiewei Fang
74086189d3 [test](tvf) move p2 tvf tests from p2 to p0 (#36871) (#37150) 
bp: #36871
 2024-07-02 22:37:43 +08:00
Ashin Gau
cf86eb8647 [test](migrate) move test_hive_text_complex_type from p2 to p0 (#37007) (#37123) 
bp: #37007
 2024-07-02 17:36:37 +08:00
qiye
4dcceaefea [test](ES Catalog) Add test cases for ES 5.x (#34441) (#36993) 
backport #34441
 2024-06-28 16:58:07 +08:00
Mingyu Chen
fcc26cc671 [test](migrate) move some cases from p2 to p0 (#36750)(#36787) (#36922) 
bp #36750 and #36787
 2024-06-27 20:59:50 +08:00
wuwenchi
ac0f6e75d2 [bugfix](iceberg)Read error when timestamp does not have time zone for 2.1 (#36435) 
bp: #36141
 2024-06-20 18:32:31 +08:00
wuwenchi
9e972cb0b9 [bugfix](iceberg)Fix the datafile path error issue for 2.1 (#36066) 
bp: #35957
 2024-06-08 21:51:46 +08:00
zy-kkk
a42b06a168 [branch-2.1][test](jdbc catalog) Change the db2 image address and repair test (#35967) 2024-06-06 17:21:40 +08:00
daidai
bc062a2595 [fix](orc)fix orc reader missing column. (#35735) 
## Proposed changes
bp #35583 
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-05-31 22:51:44 +08:00
苏小刚
7381cd56b0 [docker](hive) sync for hive initializing (#35479) 
Add healthy checking for hive2 and hive3
 2024-05-29 15:03:06 +08:00
Qi Chen
68eda58a8c [Fix](multi-catalog) Fix string dict filtering when use null related function in parquet and orc reader. (#35335) 
The following sql and when the dictionary column contains functions related to null, the results will be incorrect.
```
select * from ( select IF(o_orderpriority IS NULL, 'null', o_orderpriority) AS o_orderpriority from test_string_dict_filter_orc ) as A where o_orderpriority = 'null';
```
```
select * from ( select IFNULL(o_orderpriority, 'null') AS o_orderpriority from test_string_dict_filter_parquet ) as A where o_orderpriority = 'null'
```
```
select * from ( select COALESCE(o_orderpriority, 'null') AS o_orderpriority from test_string_dict_filter_parquet ) as A where o_orderpriority = 'null';
```
 2024-05-27 15:25:29 +08:00
zgxme
bc70968019 [chore](regression) Modify character encoding to be consistent with Doris (#35228) 2024-05-22 20:04:50 +08:00
苏小刚
4ecc3edc21 [test](hive)revert hive container to host mode (#34322) 
Revert hive container to host mode to fix pipeline problem
 2024-05-07 10:36:01 +08:00
Qi Chen
99af54f779 [Fix](orc-reader) Fix the issue when string col has mixed plain and dict encoding in different stripes. (#34146) (#34248) 
backport #34146
 2024-04-28 19:43:57 +08:00
Qi Chen
acc2b532e7 [Test](hive-writer) Adjust test_hive_write_partitions regression test to resolve special characters issue with git on windows. (#34026) 2024-04-26 15:05:47 +08:00
Mingyu Chen
7f4b7b04ad [test](hive)add subnet for hive docker compose (#34000) (#34157) 
bp #34000
Co-authored-by: 苏小刚 <suxiaogang223@icloud.com>
 2024-04-26 13:49:33 +08:00
Mingyu Chen
50f9d47e96 [test](hive) run suite cases both in hive2 and hive3 (#33874) (#34156) 
bp #33874

Co-authored-by: 苏小刚 <suxiaogang223@icloud.com>
 2024-04-26 13:48:09 +08:00
苏小刚
1c025c0488 [docker](hive) add hive3 docker compose and modify scripts (#33115) 
add hive3 docker compose from:
big-data-europe/docker-hive#56
 2024-04-17 23:42:13 +08:00
Qi Chen
4963d60a07 [Fix](multi-catalog)Fix the issue of not initializing the writer caused by refactoring and add hive writing regression test. (#32721) (#33446) 
backport #32721.
 2024-04-10 11:42:22 +08:00
Mingyu Chen
73de61ed84 [opt](hive) skip hidden file and dir (#32412) 
When query hive table, we should skip all hidden dirs and files, like:
```
/visible/.hidden/path
/visible/.hidden.txt
```
 2024-03-21 14:07:24 +08:00
wuwenchi
926908ece2 [fix](hive) fix spelling mistakes for "separatorChar" #32061 2024-03-12 14:20:18 +08:00
zy-kkk
2e9bd268cd [improvement](jdbc catalog) support sqlserver timestamp type read (#31805) 2024-03-06 13:08:04 +08:00
Ashin Gau
248ea20901 Revert "[test](regression) add regression test for schange change of complex …" (#31660) 
This reverts commit dcd2afdb4e857791fed66a46f28ab3adc25494e1.
Reverts #31207
 2024-03-01 19:06:59 +08:00
zy-kkk
07224686ef [feature](jdbc catalog) support db2 jdbc catalog (#31627) 2024-03-01 14:19:28 +08:00
zy-kkk
3c37fb085c [refactor](jdbc catalog) split jdbc executor for different data sources (step-1) (#31406) 2024-02-29 12:38:03 +08:00
Ashin Gau
e3b4b83bca [test](regression) add regression test for schange change of complex type (#31207) 
Add regression test for #31128
 2024-02-22 19:50:07 +08:00
zy-kkk
b3ac2128dd [Refactor](catalog) Refactor Jdbc Catalog external name case mapping rules (#28414) 2024-02-19 17:22:03 +08:00
Guangdong Liu
8db2824c44 [bugfix](es catalog) add constant_keyword wildcard data type (#30947) 2024-02-19 17:20:21 +08:00
Qi Chen
92cad69fc4 [Fix](parquet-reader) Fix reading fixed length byte array decimal in parquet reader. (#30535) 2024-01-31 23:53:40 +08:00
zhangdong
658c869aac [improvement](mtmv)mtmv support partition by hms table (#29989) 2024-01-29 19:02:46 +08:00
qiye
fac0580eae [opt](docker)optimize ES docker compose (#30068) 
1. add volume for es logs
2. optimize health check, waiting for es status to be green
3. fix es6 valume path error
4. optimize disk watermark to avoid es disk watermark error
5. fix es6 create index error
6. add custom elasticsearch.yml for es6
7. add log4j2.properties for es6, es7, es8
 2024-01-19 15:48:56 +08:00
wuwenchi
44ba9e102c [feature](statistics)support statistics for iceberg/paimon/hudi table (#29868) 2024-01-18 12:03:07 +08:00
Dongyang Li
67986a8a6f [fix](case) external case, specify networks (#29699) 
Co-authored-by: stephen <hello-stephen@qq.com>
 2024-01-12 11:46:29 +08:00
zy-kkk
3cd1c7745a [fix](jdbc catalog) Fix the precision of decimal type mapping to 0 (#29407) 2024-01-12 11:39:57 +08:00
qiye
2c4e52e44e [fix](es catalog) only es_query function can push down to ES (#29320) 
Issue Number: close #29318 
1. Only push down `es_query` function to ES
2. Add null check where ES query result not have `_source` or `fields` fields.
 2023-12-30 09:33:26 +08:00
wuwenchi
7da86c37ec [fix](hive) add support for quoteChar and seperatorChar for hive (#28613) 
add support for quoteChar and seperatorChar .
 2023-12-19 19:35:03 +08:00
bobhan1
01c94a554d [fix](autoinc) Fix broker load when target table has autoinc column (#28402) 2023-12-14 18:02:54 +08:00
Jibing-Li
a271fee3c5 [test](statistics)Add external empty table test case. (#28267) 2023-12-13 21:48:01 +08:00