database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
17,804 Commits 1 Branch 0 Tags
ca64fa795432a46d1869722755a67d68a1cfccee
Commit Graph

7314 Commits

Author SHA1 Message Date
Pxl
ca64fa7954 [Bug](materialized-view) do not check key/value column when index is dup or mow (#32695) 
do not check key/value column when index is dup or mow
 2024-03-27 02:56:11 +08:00
Pxl
f579eceb34 [Improvementation](profile) add some profile on vcollect_iterator (#32794) 
add some profile on vcollect_iterator
 2024-03-26 20:33:16 +08:00
airborne12
0122b8a6b4 [Update](inverted index) add config for inverted index query cache shards (#32666) 2024-03-26 20:27:33 +08:00
zhangstar333
0a44de67bf [bug](distinct agg) fix distinct streaming agg not output all data (#32760) 
fix distinct streaming agg not output all data
 2024-03-26 20:19:36 +08:00
Mryange
ad2d20348a [fix](pipeline) fix use error row desc when origin block clear #32803 (#32849) 
* fix

* add case
 2024-03-26 20:02:46 +08:00
Pxl
bda35f9ff0 [Bug](runtime-filter) fix unknown filter on nested loop join sink #32851 2024-03-26 19:10:31 +08:00
airborne12
cf24b571a3 [Pick 2.1](inverted index) fix writer searcher cache fd leak(#32564) #32812 2024-03-26 13:58:36 +08:00
lihangyu
45245209ef [Serde](Variant) support arrow serialization for varint type (#32809) 2024-03-26 10:50:13 +08:00
yiguolei
7b94cfdba1 Revert "[Fix](tests) add regression tests for trino-connector (#32552)" 
This reverts commit 3fc3a4650681cb519405730899a2f22f268b38c1.
 2024-03-25 22:38:21 +08:00
TengJianPing
ff0da8108b [fix](RF) fix 'Invalid value' error of RF of decimal type (#32749) 2024-03-25 22:34:19 +08:00
Gabriel
41b9556a25 [pipelineX](localshuffle) Adjust local exchange strategy for distinct streaming agg (#32767) 
Adjust local exchange strategy for distinct streaming agg
 2024-03-25 22:33:36 +08:00
Mryange
de3b99be00 [fix](pipeline) fix check failed in StatefulOperator 2024-03-25 22:33:30 +08:00
Tiewei Fang
3fc3a46506 [Fix](tests) add regression tests for trino-connector (#32552) 2024-03-25 22:31:55 +08:00
yiguolei
ae3542f552 [bugfix](use after free) should not set finish depdency any more if task ctx lock failed (#32730) 
Co-authored-by: yiguolei <yiguolei@gmail.com>
 2024-03-25 22:31:55 +08:00
yiguolei
e71e1b6f30 [enhancement](threadname) change thread name more simple in order to see it in top -H (#31734) 
Co-authored-by: yiguolei <yiguolei@gmail.com>
 2024-03-25 22:31:55 +08:00
Jerry Hu
bff45cf94f [fix](union) union source operator hold too many blocks from children (#32717) 2024-03-25 18:09:41 +08:00
yiguolei
55ae41000f [enhancement](fatal) change log fatal to throw exception to avoid core (#32715) 
Co-authored-by: yiguolei <yiguolei@gmail.com>
 2024-03-24 12:08:57 +08:00
Xin Liao
5c3ca0fbc2 [fix](move-memtable) fix load timeout caused by lost wakeup (#32720) 2024-03-24 08:07:01 +08:00
zhangstar333
4ebbabf15e [test](fuzzy) test fuzzy in BE (#31607) 
test fuzzy in BE
 2024-03-24 08:06:13 +08:00
zhangstar333
a55b5ea9ca [bug](udaf) fix memory leak in the java udaf (#32630) 
fix memory leak in the java udaf
 2024-03-24 08:06:13 +08:00
HappenLee
8b34915518 [Fix](compress) Fix occasional crushes when serializing blocks (#32672) 2024-03-23 06:20:45 +08:00
TengJianPing
35e580ec7a [fix](RF) fix 'Invalid value' error of RF of datetimev2 type for max value (#32649) 2024-03-22 22:29:50 +08:00
lihangyu
62c7d0a421 [Fix](point query) add query options for short circuit queries (#32530) (#32684) 
Some options like `be_exec_version` needed for functions
 2024-03-22 18:03:18 +08:00
lihangyu
fdc19b4892 [Fix](Variant) Initialize original_tablet_schema in _expand_variant_to_subcolumns to address potential nullptr issue (#32184) (#32678) 2024-03-22 18:02:58 +08:00
yuxuan-luo
3f093627f2 [fix](metrics) fix compaction_used_permits are negative numbers (#32440) 
Co-authored-by: hugoluo <hugoluo@tencent.com>
 2024-03-22 16:38:52 +08:00
wangbo
326a264fcd [Improvement](executor)Add spill property for workload group #32554 2024-03-22 16:38:19 +08:00
zhangstar333
e41311d77d [bug](fold) fix fold constant core dump with variant type (#32265) 
1. variant type core dump at call get_data_at function, as not impl this function.
2. some case can't pass at old planner and fold_constant_by_be = on.
3. open enable_fold_constant_by_be = true.
 2024-03-22 16:37:33 +08:00
苏小刚
2f2d488668 [opt](parquet) Support hive struct schema change (#32438) 
Followup: #31128
This optimization allows doris to correctly read struct type data after changing the schema from hive.

## Changing  struct schema  in hive:
```sql
hive> create table struct_test(id int,sf struct<f1: int, f2: string>) stored as parquet;

hive> insert into struct_test values
    >           (1, named_struct('f1', 1, 'f2', 's1')),
    >           (2, named_struct('f1', 2, 'f2', 's2')),
    >           (3, named_struct('f1', 3, 'f2', 's3'));

hive> alter table struct_test change sf sf struct<f1:int, f3:string>;

hive> select * from struct_test;
OK
1	{"f1":1,"f3":null}
2	{"f1":2,"f3":null}
3	{"f1":3,"f3":null}
Time taken: 5.298 seconds, Fetched: 3 row(s)
```

The previous result of doris was:
```sql
mysql> select * from struct_test;
+------+-----------------------+
| id   | sf                    |
+------+-----------------------+
|    1 | {"f1": 1, "f3": "s1"} |
|    2 | {"f1": 2, "f3": "s2"} |
|    3 | {"f1": 3, "f3": "s3"} |
+------+-----------------------+
```

Now the result is same as hive:

```sql
mysql> select * from struct_test;
+------+-----------------------+
| id   | sf                    |
+------+-----------------------+
|    1 | {"f1": 1, "f3": null} |
|    2 | {"f1": 2, "f3": null} |
|    3 | {"f1": 3, "f3": null} |
+------+-----------------------+
```
 2024-03-22 16:35:47 +08:00
Gabriel
647a0606aa [pipelineX](refactor) Wait for 2-phase execution before opening (#32613) 
Wait for 2-phase execution before opening
 2024-03-22 16:35:47 +08:00
Jerry Hu
66336e59e6 [fix](join) the result of left semi join with empty right side should be false, not null (#32477) 2024-03-22 16:35:43 +08:00
Mryange
baf3ae1a93 [refactor](nereids)unify outputTupleDesc and projection be part (#32439) 2024-03-22 16:35:43 +08:00
deardeng
ab467f53db [fix](partition) Fix be tablet partition id eq 0 By report tablet (#32179) (#32667) 2024-03-22 15:38:58 +08:00
qiye
ea71472d64 [fix](build index) fix core when build index for a new column which without data (#32550) (#32669) 
Co-authored-by: Luennng <luennng@gmail.com>
Co-authored-by: Tanya-W <tanya1218w@163,com>
 2024-03-22 15:05:19 +08:00
qiye
a4a191fe56 [fix](index compaction)Fix MOW index compaction core (#32121) (#32657) 2024-03-22 14:20:19 +08:00
Jerry Hu
23c12fd68f [fix](join) core caused by null-safe-equal join (#32623) 2024-03-22 08:53:47 +08:00
Xinyi Zou
921fab2196 [fix](memory) Fix thread context not initialized in MacOS (#32570) 2024-03-22 08:53:47 +08:00
yiguolei
6b54171778 [bugfix](deadlock) pipelinex map lock should only scope in map not about pipelinectx's cancel method (#32622) 
both global lock in fragment mgr should only protect the map logic, could not use it to protect cancel method.
fragment ctx cancel method should be protected by a lock.
query ctx cancel --> pipelinex fragment cancel ---> query ctx cancel will dead lock.
 2024-03-22 08:52:38 +08:00
Pxl
6462d913ca [Improvement](brpc) log error message when AutoReleaseClosure meet brpc error or response… (#32628) 
log error message when AutoReleaseClosure meet brpc error or response with error status
 2024-03-22 08:52:38 +08:00
zhannngchen
d3bdda6071 [fix](partial update) fix data correctness risk when load delete sign data into a table with sequence col (#32574) 2024-03-22 08:52:38 +08:00
Kang
55b7f7f019 [fix](inverted index) skip read index column data only for DUP and MOW table (#32594) 2024-03-22 08:52:16 +08:00
amory
2cb652a7fa [FIX](compile)fix for gcc compile (#32508) 
* fix for gcc compile
 2024-03-22 08:52:16 +08:00
Tiewei Fang
d7a3ff1ddf [Fix](Outfile) Fix the column type mapping in the orc/parquet file format (#32281) 
| Doris Type             | Orc Type                     |  Parquet Type                |
|---------------------|--------------------|------------------------|
| Date                            | Long (logical: DATE)                 |       int32 (Logical: Date)                                        |
| DateTime                    | TIMESTAMP (logical: TIMESTAMP)    |       int96                          |
 2024-03-22 08:52:16 +08:00
zy-kkk
fd0bc720e9 [opt](information_schema) Add DEFAULT_ENCRYPTION column to schemata table (#32501) 2024-03-22 08:52:16 +08:00
Gabriel
6888e52365 [pipelineX](fix) Fix illegal memory access (#32602) 2024-03-22 08:52:16 +08:00
Jerry Hu
844dd8b2ce [fix](spill) should wait for merging done before read agg result (#32537) 2024-03-22 08:52:16 +08:00
abmdocrt
fd62af82d2 [enhancement](mow) Add bvar for bloom filter and segment (#32355) 2024-03-22 08:52:12 +08:00
zzzxl1993
0cde0cbf19 (invert index) modify of time series compaction policy 2024-03-22 08:16:30 +08:00
Mingyu Chen
4c8aaa156a [fix](jni) remove 'push_down_predicates' and fix BE crash with decimal predicate (#32253) (#32599) 2024-03-21 14:07:50 +08:00
lihangyu
617cc667fe [Fix](Variant) fix variant serialize root node (#31769) 2024-03-21 14:07:50 +08:00
Gabriel
02ef02402a [pipelineX](debug) Add debug logs for long-running load task (#32534) 2024-03-21 14:07:50 +08:00