database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
17,731 Commits 1 Branch 0 Tags
8a6fc79797de660d4dd4e6db971e86f641eecbaa
Commit Graph

17731 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
HHoflittlefish777
8a6fc79797 [fix](routine-load) avoid routine load pause for check transaction status fail (#32638) 2024-03-22 16:36:49 +08:00
Gabriel
d30ad2ba04 [test](regression) delete invalid test cases (#32654) 2024-03-22 16:36:49 +08:00
924060929
6812b575b2 [fix](Nereids) fix bind having aggregate failed (#32490) 
fix bind having aggregate failed, keep the behavior like mysql
 2024-03-22 16:36:46 +08:00
xueweizhang
1c521cd94e [fix](backup) clear snapshotInfos and backupMeta when cancel (#32646) 2024-03-22 16:36:46 +08:00
zy-kkk
a10466598b [fix](jdbc catalog) Fix query errors without jdbc pool default value on only BE upgrade (#32618) 2024-03-22 16:36:22 +08:00
苏小刚
2f2d488668 [opt](parquet) Support hive struct schema change (#32438) 
Followup: #31128
This optimization allows doris to correctly read struct type data after changing the schema from hive.

## Changing  struct schema  in hive:
```sql
hive> create table struct_test(id int,sf struct<f1: int, f2: string>) stored as parquet;

hive> insert into struct_test values
    >           (1, named_struct('f1', 1, 'f2', 's1')),
    >           (2, named_struct('f1', 2, 'f2', 's2')),
    >           (3, named_struct('f1', 3, 'f2', 's3'));

hive> alter table struct_test change sf sf struct<f1:int, f3:string>;

hive> select * from struct_test;
OK
1	{"f1":1,"f3":null}
2	{"f1":2,"f3":null}
3	{"f1":3,"f3":null}
Time taken: 5.298 seconds, Fetched: 3 row(s)
```

The previous result of doris was:
```sql
mysql> select * from struct_test;
+------+-----------------------+
| id   | sf                    |
+------+-----------------------+
|    1 | {"f1": 1, "f3": "s1"} |
|    2 | {"f1": 2, "f3": "s2"} |
|    3 | {"f1": 3, "f3": "s3"} |
+------+-----------------------+
```

Now the result is same as hive:

```sql
mysql> select * from struct_test;
+------+-----------------------+
| id   | sf                    |
+------+-----------------------+
|    1 | {"f1": 1, "f3": null} |
|    2 | {"f1": 2, "f3": null} |
|    3 | {"f1": 3, "f3": null} |
+------+-----------------------+
```
 2024-03-22 16:35:47 +08:00
morrySnow
01a5413e45 [fix](Nereids) filter-limit-project translate to wrong plan (#32496) 2024-03-22 16:35:47 +08:00
xueweizhang
b2db5a4ded [fix](restore) fix restore odbc resource bug (#31989) 
if some odbc tables use the same resource, and restore not all odbc tables, it will not retain the resource.
and check some conf for backup/restore

Signed-off-by: nextdreamblue <zxw520blue1@163.com>
 2024-03-22 16:35:47 +08:00
谢健
4de8775e17 [feat](Nereids): rewrite sum literal to sum and count (#32244) 
sum(v + 2) => sum(v) + 2*count(v)
sum(v - 2) => sum(v) - 2*count(v)
 2024-03-22 16:35:47 +08:00
谢健
8f3f9a53be [feat](Nereids): add is null predicate for the first partition when updating mv by partition (#32463) 2024-03-22 16:35:47 +08:00
morrySnow
3f36aa2d48 [chore](Nereids) remove ensure project on top join (#32562) 2024-03-22 16:35:47 +08:00
starocean999
4b34ecefbe [fix](nereids)str_to_date function's signature for folding constant is wrong (#32474) 2024-03-22 16:35:47 +08:00
Gabriel
647a0606aa [pipelineX](refactor) Wait for 2-phase execution before opening (#32613) 
Wait for 2-phase execution before opening
 2024-03-22 16:35:47 +08:00
jakevin
39382a9774 [fix](Nereids): just pull up alias project above join through topn (#32305) 2024-03-22 16:35:47 +08:00
Jerry Hu
66336e59e6 [fix](join) the result of left semi join with empty right side should be false, not null (#32477) 2024-03-22 16:35:43 +08:00
Mryange
baf3ae1a93 [refactor](nereids)unify outputTupleDesc and projection be part (#32439) 2024-03-22 16:35:43 +08:00
deardeng
ab467f53db [fix](partition) Fix be tablet partition id eq 0 By report tablet (#32179) (#32667) 2024-03-22 15:38:58 +08:00
yiguolei
0f343e914a Revert "[case](Cloud) Add ssb case for hdfs vault (#32567)" 
This reverts commit 60a673979e5b2f98e9cc66c77fa0d6c61b8ed0f7.
 2024-03-22 15:26:09 +08:00
seawinde
ebfb3418f9 [test](mtmv)Add tpch test cherry pick to branch 21 (#32611) 
* [test](neredis) Add tpch test for query rewrite by materialized view (#30870)

query rewrite by materialized view sql is as following
q1, q5, q6, q8, q9, q12, q14
the other is not supported now, will be supported later

* change code usage
 2024-03-22 15:20:38 +08:00
qiye
ea71472d64 [fix](build index) fix core when build index for a new column which without data (#32550) (#32669) 
Co-authored-by: Luennng <luennng@gmail.com>
Co-authored-by: Tanya-W <tanya1218w@163,com>
 2024-03-22 15:05:19 +08:00
qiye
a4a191fe56 [fix](index compaction)Fix MOW index compaction core (#32121) (#32657) 2024-03-22 14:20:19 +08:00
yiguolei
2e25b0079b fix compile 2024-03-22 09:17:40 +08:00
Jerry Hu
23c12fd68f [fix](join) core caused by null-safe-equal join (#32623) 2024-03-22 08:53:47 +08:00
Xinyi Zou
921fab2196 [fix](memory) Fix thread context not initialized in MacOS (#32570) 2024-03-22 08:53:47 +08:00
walter
e5d7ff64aa [fix](backup) clear snapshot info for cancelled backup job to reduce log size (#32604) 2024-03-22 08:53:17 +08:00
yiguolei
6b54171778 [bugfix](deadlock) pipelinex map lock should only scope in map not about pipelinectx's cancel method (#32622) 
both global lock in fragment mgr should only protect the map logic, could not use it to protect cancel method.
fragment ctx cancel method should be protected by a lock.
query ctx cancel --> pipelinex fragment cancel ---> query ctx cancel will dead lock.
 2024-03-22 08:52:38 +08:00
Pxl
6462d913ca [Improvement](brpc) log error message when AutoReleaseClosure meet brpc error or response… (#32628) 
log error message when AutoReleaseClosure meet brpc error or response with error status
 2024-03-22 08:52:38 +08:00
zhannngchen
d3bdda6071 [fix](partial update) fix data correctness risk when load delete sign data into a table with sequence col (#32574) 2024-03-22 08:52:38 +08:00
Kang
55b7f7f019 [fix](inverted index) skip read index column data only for DUP and MOW table (#32594) 2024-03-22 08:52:16 +08:00
xzj7019
26efde2925 [test](Nereids) refine group by eliminate regression test (#32543) 2024-03-22 08:52:16 +08:00
Mingyu Chen
06026b2f46 [opt](paimon) add split num and partition num (#32597) 
1. And selected partition number indicator in PaimonScanNode in explain result.
2. Add raw file split and total split indicator in PaimonScanNode in explain result.
3. Opt the error msg of table valued function, print the error msg first to avoid the long msg be truncated.
 2024-03-22 08:52:16 +08:00
zy-kkk
8d6561b32d [opt](jdbc catalog) set jdbc test_connection default true (#32546) 2024-03-22 08:52:16 +08:00
zhangdong
60eeff8e18 [enhance](mtmv)refresh mtmv must add auto (#32522) 2024-03-22 08:52:16 +08:00
wangbo
e7919ab1b6 Add DISABLE_BUILD_UI/DISABLE_BUILD_SPARK_DPP/DISABLE_BUILD_HIVE_UDF in build.sh (#32595) 2024-03-22 08:52:16 +08:00
amory
2cb652a7fa [FIX](compile)fix for gcc compile (#32508) 
* fix for gcc compile
 2024-03-22 08:52:16 +08:00
Tiewei Fang
d7a3ff1ddf [Fix](Outfile) Fix the column type mapping in the orc/parquet file format (#32281) 
| Doris Type             | Orc Type                     |  Parquet Type                |
|---------------------|--------------------|------------------------|
| Date                            | Long (logical: DATE)                 |       int32 (Logical: Date)                                        |
| DateTime                    | TIMESTAMP (logical: TIMESTAMP)    |       int96                          |
 2024-03-22 08:52:16 +08:00
zhangdong
ca05129fa1 [fix](auth)node priv can login web ui (#32521) 2024-03-22 08:52:16 +08:00
zhangdong
c1d42eaa86 [fix](mtmv)resolve the issue of table version updates in concurrent situations (#32487) 
Move the logic for version+1 from `unprotectedCommitTransaction `to`FinishTransaction`, as the write lock for the table was obtained in `FinishTransaction`
 2024-03-22 08:52:16 +08:00
Jibing-Li
fe1b7a7b0a Improve analyze stats case, avoid cluster delay caused failure. (#32507) 2024-03-22 08:52:16 +08:00
zy-kkk
fd0bc720e9 [opt](information_schema) Add DEFAULT_ENCRYPTION column to schemata table (#32501) 2024-03-22 08:52:16 +08:00
Gabriel
6888e52365 [pipelineX](fix) Fix illegal memory access (#32602) 2024-03-22 08:52:16 +08:00
wangbo
886aabda31 [Fix](executor) Fix insert select not close coordinator #32571 2024-03-22 08:52:16 +08:00
Jerry Hu
844dd8b2ce [fix](spill) should wait for merging done before read agg result (#32537) 2024-03-22 08:52:16 +08:00
abmdocrt
0537952573 [Fix](auto-inc) Fix auto inc unique table case (#32393) 2024-03-22 08:52:16 +08:00
abmdocrt
fd62af82d2 [enhancement](mow) Add bvar for bloom filter and segment (#32355) 2024-03-22 08:52:12 +08:00
slothever
b727fd6594 (kerberos)fix hive keberos principal usage 2024-03-22 08:16:59 +08:00
jackwener
4e653b0baa (Nereids): remove test infer_set_operator_distinct(CBO) 2024-03-22 08:16:49 +08:00
zzzxl1993
0cde0cbf19 (invert index) modify of time series compaction policy 2024-03-22 08:16:30 +08:00
Mingyu Chen
4c8aaa156a [fix](jni) remove 'push_down_predicates' and fix BE crash with decimal predicate (#32253) (#32599) 2024-03-21 14:07:50 +08:00
zfr95
a7fa9f290c [fix](test)change stream load of auto partition (#32544) 2024-03-21 14:07:50 +08:00