eb3e0b2f7d
[test](Nereids): add more plan equals test for Nereids ( #12127 )
...
- add more plan equals test for Nereids
- fix join equals bugs
2022-08-29 11:46:30 +08:00
ac425d4bf3
[fix](remote)Fix bug for cache reader ( #12104 )
2022-08-29 11:28:17 +08:00
fe9767941d
[fix](array-type) adjust enable_array_type config ( #12071 )
...
Problem:
1. `enable_array_type` is masterOnly;
2. dynamic open config only affect FE MASTER
`admin set frontend config("enable_array_type"="true");`
3. query in FE FOLLOWER will fail, because of `enable_array_type` is false in FE FOLLOWER
`select * from table_with_array `
Solution:
Only check `enable_array_type` while creating new tables with array column.
Co-authored-by: cambyzju <zhuxiaoli01@baidu.com >
2022-08-29 11:10:52 +08:00
44c4a45f72
[fix](array-type) fix the wrong data when use stream load to import '\N' ( #12102 )
...
Co-authored-by: hucheng01 <hucheng01@baidu.com >
2022-08-29 09:53:37 +08:00
7fbcf3c8ba
[api-change](http) change kill query http api by using query id ( #12120 )
...
Now user can cancel query id by http by following steps:
Get query id by trace id
cancel query by query id
The modified api has not been released yet.
2022-08-29 09:51:51 +08:00
dec576a991
[feature-wip](parquet-reader) generate null values and NullMap for parquet column ( #12115 )
...
Generate null values and NullMap for the nullable column by analyzing the definition levels.
2022-08-29 09:30:32 +08:00
f26c051835
[Typo](doc) Rewrite the Quick Start document ( #11956 )
2022-08-29 09:25:58 +08:00
acd7ab379d
[feature](Nereids)add range partition prune ( #11964 )
...
1. Rewrite Filter(Project) to Project(Filter) to make sure when do partition prune the tree looks like this: Project(Filter(OlapScan)).
2. Enable the MergeConsecutiveProject MergeConsecutiveFilter rules.
3. prune range partition just like what Legacy Planner do.
2022-08-28 23:39:09 +08:00
6e6269c682
[Improvement](load) accelerate streamload and compaction ( #12119 )
...
* [Improvement](load) accelerate streamload and compaction
2022-08-28 23:10:47 +08:00
2f88dfd7c5
[regression-test](unique) Add tpch sf0.1 test under unique table with zstd ( #12122 )
2022-08-27 21:39:09 +08:00
5693fdaf9c
fix configs default value in docs ( #12047 )
...
fix configs default value in docs
2022-08-27 10:08:54 +08:00
bd4f994c7e
update document of alter table oartition ( #12077 )
...
update document of alter table oartition
2022-08-27 10:08:13 +08:00
8c5005f247
[typo](doc)Add the backup operation, using the following instructions from S3 ( #12091 )
...
Add the backup operation, using the following instructions from S3
2022-08-27 10:07:15 +08:00
a6e2e2f3bc
[feature](remote)Add cache files cleaner for remote olap files ( #11959 )
2022-08-26 23:59:36 +08:00
0b5bb565a7
[feature-wip](parquet-reader) parquet dictionary decoder ( #11981 )
...
Parse parquet data with dictionary encoding.
Using the PLAIN_DICTIONARY enum value is deprecated in the Parquet 2.0 specification.
Prefer using RLE_DICTIONARY in a data page and PLAIN in a dictionary page for Parquet 2.0+ files.
refer: https://github.com/apache/parquet-format/blob/master/Encodings.md
2022-08-26 19:24:37 +08:00
818afa4c66
[fix](Nereids)fix some problem in CBO. ( #12083 )
...
This PR fix three problem in Nereids.
- Add selected index, partition and tablet Info in LogicalOlapScan
- with JoinReorderContext in new LogicalJoin
- fix compute data size when no column size info in StatsCaculator
2022-08-26 18:52:59 +08:00
f3f17eb222
[Bugfix](load) fix be will coredump when parsing malformed json file using simdjson ( #12062 )
...
* [Bugfix](load) fix be will coredump when parsing malformed json file using simdjson
2022-08-26 18:01:19 +08:00
fba2658a1d
[fix](array-type) fix the be core dump when use collect_list result to insert ( #12045 )
...
Co-authored-by: hucheng01 <hucheng01@baidu.com >
2022-08-26 18:00:43 +08:00
2b3a5b5fdd
[fix](array-type) add ARRAY_BOOLEAN support for lots of array functions #12079
...
Co-authored-by: cambyzju <zhuxiaoli01@baidu.com >
2022-08-26 18:00:29 +08:00
f6023d8a5b
[sample](flink-connector) add full db sync v2 ( #12090 )
...
* add db sync v2
2022-08-26 16:13:01 +08:00
4636d6c476
[enhancement](Nereids)merge consecutive LogicalLimit plan ( #12096 )
...
This rule aims to merge consecutive limits.
LogicalLimit(limit=10, offset=4)
+---LogicalLimit(limit=3, offset=5)
transformed to
LogicalLimit(limit=3, offset=5)
where
newLimit.limit = min(topLimit.limit, bottomLimit.limit)
newLimit.offset = bottomLimit.offset
topLimit.offset is ignored
2022-08-26 15:53:44 +08:00
3af0745c8f
[Bug](function) fix aggFnParams set not correct ( #12006 )
2022-08-26 14:29:56 +08:00
7b83308ffc
[fix](planner) fix rewrite subquery error ( #12075 )
2022-08-26 14:29:14 +08:00
22157077e9
[fix](memtracker) Optimize the return msg of process memory limit exceed #12086
...
Return the real process memory information when the process exceeds mem limit
Optimize the memory exceed limit log printing logic
process tracker does not participate in process memory limit.
2022-08-26 14:28:46 +08:00
9caaa4bfbd
[fix](memory) fix set disable_chunk_allocator_in_vec=false performance #12092
2022-08-26 14:28:12 +08:00
89d6f1231f
[fix](projection)join node should always output at least one column ( #12080 )
2022-08-26 12:15:51 +08:00
ccff3f5711
[bugfix](light weight schema change) support delete condition in schema change ( #11869 )
...
* [bugfix](light weight schema change) support delete condition in schema change
Co-authored-by: yiguolei <yiguolei@gmail.com >
2022-08-26 11:45:55 +08:00
e478f096ed
[typo](doc) fix some typo in docs #12054
2022-08-26 11:44:22 +08:00
82ca62dfcc
[fix](memory) Fix disable_mem_pools to disable cache #12087
2022-08-26 11:43:19 +08:00
0f4a1e811b
[Enhancement](table_function) table function node enhancement ( #12038 )
...
* table function node enhancement
* also avoid copy for non-vec table function node
* fix table function node output slots calculation while lateral view involves subquery
Co-authored-by: cambyzju <zhuxiaoli01@baidu.com >
2022-08-26 10:37:15 +08:00
ba11d8dc67
[feature-wip](unique-key-merge-on-write) fix bugs on tablet clone #12067
2022-08-26 10:37:00 +08:00
de148465bc
[feature-wip](unique-key-merge-on-write) unique key table with MOW supports update ( #11882 )
2022-08-26 09:44:37 +08:00
0c16740f5c
[feature-wip](parquet-reader) parquert scanner can read data ( #11970 )
...
Co-authored-by: jinzhe <jinzhe@selectdb.com >
2022-08-26 09:43:46 +08:00
721d418a2f
[feature-wip](unique-key-merge-on-write) fix that version is awlays 0 when update delete bitmap ( #12044 )
2022-08-26 09:41:55 +08:00
0bacbb95c2
[typo](doc)website sidebar Menu order ( #12073 )
...
menu order
2022-08-26 09:41:45 +08:00
e5bfbbe761
[feature-wip](unique-key-merge-on-write) support alter table column for MoW ( #12052 )
2022-08-26 09:40:11 +08:00
8deead66cd
[feature-wip](unique-key-merge-on-write) add pk_uk correctness check case ( #12060 )
...
Co-authored-by: yixiutt <yixiu@selectdb.com >
2022-08-26 09:38:38 +08:00
17b809210a
[Bug](runtime filter) fix bug for late-arrival runtime filters ( #12049 )
2022-08-26 09:13:10 +08:00
98d82915d3
[fix](bdbje) add reserved disk config to avoid too many reserved bdbje files ( #12043 )
2022-08-26 08:12:27 +08:00
2e216b9359
[fix](ut) fix ctas regression test case ( #12076 )
2022-08-26 08:12:11 +08:00
e3ab2caef8
[improvement](sink) Support local exchange for multi fragment instances ( #12017 )
2022-08-25 19:28:23 +08:00
f19c344328
[behavior change](planner)change Doris's query organization syntax to standard sql ( #9745 )
2022-08-25 17:17:13 +08:00
858cb74bef
[typo](doc)Window funnel fix ( #12064 )
...
window funnel fix
2022-08-25 15:46:47 +08:00
588dc5f12a
[feature](cold_on_s3) Show remote data usage via SHOW BACKENDS and SHOW TABLETS statements ( #11450 )
2022-08-25 15:36:15 +08:00
4217b9c1d3
[feature] (dbt) add incremental and init interactive command line ( #11870 )
...
add dbt-doris incremental model and init interactive command line
2022-08-25 15:03:28 +08:00
aec24d4da1
[feature](http) get create table stmt for a given query ( #11979 )
...
This API can help user to get all create table statement of a given SQL.
So that it will be easier to run the case in other Doris cluster.
See document for details
2022-08-25 15:02:05 +08:00
003fdf2b36
[fix](scan) use serial scan thread token only for scan node ( #12058 )
...
Only the scan node's limit is less than 1024, we can use serial thread token to submit scanners.
Or it will slow down the query.
2022-08-25 14:54:02 +08:00
9d165797c5
[ci](shellcheck)shellcheck include ssb-tools and tpch-tools ( #12039 )
2022-08-25 11:52:13 +08:00
620d33a763
[Enchancement](optimize) set result_size_hint to filter_block ( #11972 )
2022-08-25 11:42:52 +08:00
b505d6cf66
[improvement](doc test) add docs for window_funnel and refine tests ( #12059 )
...
add docs for window_funnel and refine tests
2022-08-25 11:26:40 +08:00
