database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
18,263 Commits 1 Branch 0 Tags
42e91149e4df989a5cbdbcdeb72b5377e58ed653
Commit Graph

18263 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Kaijie Chen
18c9ebce95 [improve](move-memtable) tweak load stream flush token num and max tasks (#28884) 2023-12-22 20:08:47 +08:00
HappenLee
fa0ad56817 [exec](compress) use FragmentTransmissionCompressionCodec control the exchange compress behavior (#28818) 2023-12-22 19:50:57 +08:00
Dongyang Li
37faf1bb80 [fix](ci) tpch pipeline should not re-load data (#28874) 
* [fix](ci) tpch pipeline should not re-load data

* 2

---------

Co-authored-by: stephen <hello-stephen@qq.com>
 2023-12-22 19:50:41 +08:00
xzj7019
b2b209e938 [feature](Nereids) Pull up join from union all (#28682) 2023-12-22 19:35:20 +08:00
airborne12
3ed82bcee2 [Feature](inverted index) add lowercase option for inverted index analyzer (#28704) 2023-12-22 18:22:44 +08:00
Gabriel
9e0a2e861c [pipelineX](refactor) rename functions (#28846) 2023-12-22 17:24:39 +08:00
wangbo
aca8406e31 [refactor](executor)remove scan group #28847 2023-12-22 17:05:50 +08:00
TengJianPing
d75300f166 [fix](hash join) fix stack overflow caused by evaluate case expr on huge build block (#28851) 2023-12-22 15:45:12 +08:00
Gabriel
cb61a0727f [bug](sharedscan) Fix shared scan bug (#28841) 2023-12-22 15:13:14 +08:00
minghong
5420575184 check stats and log memo for ds46 (#28396) 2023-12-22 14:50:11 +08:00
minghong
7f310cec9c [opt](sessionVar)show changed sessoin var first #28840 
“show variables” command list changed vars before not changed vars,
 2023-12-22 14:45:33 +08:00
zzzxl
9b67c86219 [optimize](count) optimize pk exact query without reading data (#28494) 2023-12-22 14:18:15 +08:00
AlexYue
007f498f3b (enhance)(InternalQuery) Support to collect profile for intenal query (#28762) 2023-12-22 14:03:48 +08:00
zhangdong
d1e1619e89 [feature](mtmv)mtmv partition refresh case (#28787) 2023-12-22 14:03:31 +08:00
wuwenchi
7710c85904 [fix](stream load)add test case and doc for arrow type of stream load (#28098) 
add test case and doc for arrow type of stream load
 2023-12-22 13:18:44 +08:00
wuwenchi
f38e11ec4e [fix](paimon)fix type convert for paimon (#28774) 
fix type convert for paimon
 2023-12-22 13:18:25 +08:00
Xiangyu Wang
fb52e111c5 [Enhancement](job) No need to query some backends which are not alive. (#28608) 
No need to execute some jobs if backend is not alive
 2023-12-22 13:14:30 +08:00
Xiangyu Wang
dc786114a2 [Fix](multi-catalog) skip hms events if hms table is not supported. (#28644) 
Co-authored-by: wangxiangyu <wangxiangyu@360shuke.com>
 2023-12-22 13:07:58 +08:00
zhiqiang
8c59e16f81 [opt](query cancel) optimization for query cancel #28778 2023-12-22 12:48:37 +08:00
zhiqiang
e4c7d4d03a [chore](error msg) print type info when colocate with ddl failed due to type mismatch (#28773) 2023-12-22 12:48:20 +08:00
wangbo
012e66729a [improvement](executor) Add tvf and regression test for Workload Scheduler (#28733) 
1 Add select workload schedule policy tvf
2 Add reg test
 2023-12-22 12:09:51 +08:00
Ashin Gau
c72ad9b673 [fix](regression) fix regression error of test_compress_type (#28826) 2023-12-22 12:08:23 +08:00
Xinyi Zou
83e7235bab [fix](memory) Add thread asynchronous purge jemalloc dirty pages (#28655) 
jemallctl purge all arena dirty pages may take several seconds, which will block memory GC and cause OOM.
So purge asynchronously in a thread.
 2023-12-22 12:05:20 +08:00
lihangyu
13ccfa06a7 [Feature](Variant) Implement variant new sub column access method (#28484) 
* [Feature](Variant) Implement variant new sub column access method

The query SELECT v["a"]["b"] from simple_var WHERE cast(v["a"]["b"] as int) = 1 encompasses three primary testing scenarios:

```
1. A basic test involving the variant data type.
2. A scenario dealing with GitHub event data in the context of a variant.
3. A case related to the TPC-H benchmark using a variant.
```
 2023-12-22 11:59:37 +08:00
zclllyybb
453e3c18f4 [refactor](buffer) remove download buffer since it is no longer useful (#28832) 
remove download buffer since it is no longer useful
 2023-12-22 11:53:31 +08:00
lihangyu
f6b6180462 [Fix](Variant) fix variant predicate rewrite OrToIn with wrong plan (#28695) 
using the name without paths info will lead to wrong In plan, e.g.
```
where cast(v:a as text) = 'hello' or cast(v:b as text) = 'world'
```
will be rewrite to:
```
where cast(v as text) in ('hello', 'world')
``
This is wrong, because they are different slots
 2023-12-22 11:51:36 +08:00
meiyi
0af6bd6390 [fix](group-commit) check if wal need recovery is abnormal (#28769) 2023-12-22 11:06:11 +08:00
Kaijie Chen
e2941aa9c8 [improve](config) set mutable and masterOnly in FE config stream_load_default_memtable_on_sink_node (#28835) 2023-12-22 10:58:43 +08:00
starocean999
6962de841d [fix](test)fix test_create_table test case for nereids (#28693) 2023-12-22 10:55:35 +08:00
zhannngchen
25dcf40f36 [test](partial update) add complex type regression cases for partial update (#28758) 
NOTE: There's some issue for MAP type to work with row store, so in this PR we don't have cases for MAP type
Will add the support for MAP type in future.
 2023-12-22 10:55:24 +08:00
lw112
172f68480b [Enhancement](load) Limit the number of incorrect data drops and add documents (#27727) 
In the load process, if there are problems with the original data, we will store the error data in an error_log file on the disk for subsequent debugging. However, if there are many error data, it will occupy a lot of disk space. Now we want to limit the number of error data that is saved to the disk.

Be familiar with the usage of doris' import function and internal implementation process
Add a new be configuration item load_error_log_limit_bytes = default value 200MB
Use the newly added threshold to limit the amount of data that RuntimeState::append_error_msg_to_file writes to disk
Write regression cases for testing and verification

Co-authored-by: xy720 <22125576+xy720@users.noreply.github.com>
 2023-12-22 10:43:18 +08:00
zclllyybb
0b9b1be1f1 [fix](function) Fix from_second functions overflow and wrong result (#28685) 2023-12-22 10:22:49 +08:00
bobhan1
49eaf0cc32 [fix](partial update) only report error when in strict mode partial update when finding missing rowsets during flushing memtable (#28764) 
related pr: #28062, #28674, #28677
fix #28677
 2023-12-22 09:50:10 +08:00
Jibing-Li
c1457f97a4 [improvement](external catalog)Change log level to debug for getRowCountFromExternalSource. (#28801) 2023-12-21 22:09:50 +08:00
Siyang Tang
65e3da255f [enhancement](audit-log) add status code and error msg to audit log for proxy stmt (#27607) 2023-12-21 21:16:36 +08:00
Kaijie Chen
5153137b83 [fix](metrics) fix bvar memtable_input_block_allocated_size (#28725) 2023-12-21 21:16:14 +08:00
morrySnow
e51e94fdec [fix](Nereids) join order is not right after sql parsing (#28721) 
for sql
```
t1, t2 join t3
```

we should generate plan like:
```
t1 join (t2 join t3)
```

but we generate:
```
(t1 join t2) join t3
```
to follow legancy planner.
 2023-12-21 20:31:40 +08:00
huanghaibin
e74ff95087 [fix](compaction) compaction should catch exception when vertical block reader read next block (#28625) 2023-12-21 20:30:37 +08:00
huanghaibin
0070909d30 [fix](group commit)Fix the issue of duplicate addition of wal path when encouter exception (#28691) 2023-12-21 20:27:33 +08:00
minghong
ba4033ad2e [opt](Nereids) improve Shape check cases (#28124) 
* tmplate update
* log tpcds stats when check shape
 2023-12-21 20:19:54 +08:00
Kaijie Chen
ee73833d6e [improve](load) reduce lock scope in MemTableWriter active consumption (#28790) 2023-12-21 20:18:35 +08:00
qiye
cd65796874 [opt](inverted index) ignore_above only affects untokenized strings (#28819) 2023-12-21 20:06:56 +08:00
Gabriel
619d500333 [bug](pipelineX) Fix pipelineX bug on multiple BE (#28792) 2023-12-21 19:15:52 +08:00
starocean999
a16680cf91 [fix](planner)fix bug of bound conjunct to wrong tuple (#28811) 
this fix bug introduced by #28656
 2023-12-21 19:15:36 +08:00
Yoko
4f1aebb8e8 (topN)runtime_predicate is only triggered when the column name is obtained (#28419) 
Issue Number: close #27485
 2023-12-21 18:08:23 +08:00
Gabriel
5c469a8b6c [pipelineX](fix) Fix TPCH Q2 (#28783) 2023-12-21 17:11:01 +08:00
Pxl
f3bf26c269 [Chore](decimal) set check_overflow_for_decimal to true when alter table(#28777) 
set check_overflow_for_decimal to true when alter table
 2023-12-21 16:27:06 +08:00
XuJianxu
6dcf79fc27 [Bug fix][metrics] correct fe collector type for jvm_gc (#28784) 
Co-authored-by: 胥剑旭 <xujianxu@xujianxudeMacBook-Pro.local>
 2023-12-21 16:21:13 +08:00
Mingyu Chen
4ddef31c20 (fix)[meta][export] fix replay export NPE issue (#28752) 
The ConnectionContext does not exist in replay thread
 2023-12-21 14:39:00 +08:00
Mingyu Chen
5d8c465644 [regression](p2) fix test cases result (#28768) 
regression-test/data/external_table_p2/hive/test_hive_hudi.out
regression-test/data/external_table_p2/hive/test_hive_to_array.out
regression-test/suites/external_table_p2/tvf/test_local_tvf_compression.groovy
regression-test/suites/external_table_p2/tvf/test_path_partition_keys.groovy
regression-test/data/external_table_p2/hive/test_hive_text_complex_type.out
 2023-12-21 14:38:30 +08:00