bccea1c511
[Enhancement](partition prune): calculate the column ranges of compound predicates ( #14886 )
...
Doris does not support disjunctive predicates very well, which causes some problems in partition prune.
For example, sqls like the followings will trigger a full table scan without partition pruning
select * from test.t1
where (dt between 20211121 and 20211122) or (dt between 20211125 and 20211126)
2022-12-15 20:47:44 +08:00
c4de619110
[fix](merge-on-write) calc delete bitmap need all segments which _do_flush in one memtable ( #15018 )
...
when some case(need modify be.conf), a memtable may flush many segments and then calc delete bitmap with new data. but now, it just only load one segment with max sgement id and this bug will not cala delte bitmap with all data of all segment of one memtable, and will get many rows with same key from merge-on-write table.
2022-12-15 20:44:49 +08:00
401d5776b0
[fix](compile) compile error while with DORIS_WITH_MYSQL #15105
...
Co-authored-by: cambyzju <zhuxiaoli01@baidu.com >
2022-12-15 20:40:33 +08:00
71121deed9
[typo](docs)fix fe config en doc err ( #15111 )
2022-12-15 20:27:12 +08:00
2fcdd766bd
[fix](remote) fix be crash while write data to s3 in debug mode ( #15047 )
2022-12-15 19:56:35 +08:00
9fb62a23f4
[Bug](function) fix overflow on concat_ws ( #15043 )
...
fix overflow on concat_ws
2022-12-15 19:44:27 +08:00
6625e650c4
[fix](resource) HdfsStorage can get default.Fs from path or configuration ( #15079 )
2022-12-15 16:56:32 +08:00
c6d93f739c
[feature-wip](file reader) Merge stream_load_pipe to the new file reader ( #15035 )
...
Currently, there are two sets of file readers in Doris, this pr rewrites the old stream_load_pipe with the new file reader.
2022-12-15 16:31:22 +08:00
face82e56a
[fix](meta) fix bug that backend tag may change to default after upgrading to 1.2 ( #15085 )
2022-12-15 12:07:11 +08:00
67e4292533
[fix](iceberg-v2) icebergv2 filter data path ( #14470 )
...
1. a icebergv2 delete file may cross many data paths, so the path of a file split is required as a predicate to filter rows of delete file
- create delete file structure to save predicate parameters
- create predicate for file path
2. add some log to print row range
3. fix bug when create file metadata
2022-12-15 10:18:12 +08:00
7e90fc5784
[regresion-test](config) open fe debug log ( #15044 )
2022-12-15 10:06:25 +08:00
21c2e485ae
[improvment](function) add new function substring_index ( #15024 )
2022-12-15 09:54:34 +08:00
46030d786a
[Bug](datetimev2) Fix wrong result after insertion ( #15052 )
2022-12-15 09:54:18 +08:00
03847b6a3a
[Feature](Api) Support operate node(fe/be). ( #14904 )
...
Support operate node(fe/be) via http
2022-12-14 23:18:56 +08:00
5cf88a5339
[improvement](config) opt the message when missing JAVA_HOME for BE ( #15045 )
...
Make the error message easy to understand
2022-12-14 23:17:46 +08:00
f1b2668a62
[typo](doc) Indicates that the order by feature in group_concat function is supported from version 1.2 ( #15083 )
2022-12-14 21:24:06 +08:00
05805a1632
[typo](docs)Add fe config enable_new_load_scan_node ( #15075 )
2022-12-14 18:09:53 +08:00
48a70212ac
[bugfix](BE)fix broker file compile error ( #15070 )
2022-12-14 15:46:01 +08:00
41838e6acb
[fix](string-type) rectify string type' len to MAX_STRING_LENGTH ( #14985 )
...
cherry pick from #14587
2022-12-14 15:41:08 +08:00
83e81c60a2
[fix](regression) Nereids' const by constant case is unstable ( #15062 )
2022-12-14 15:03:35 +08:00
9d6a81d1e3
[improvement](query)optimize select stmt with limit 0 ( #14956 )
2022-12-14 13:48:09 +08:00
b8f93681eb
[feature-wip](file reader) Merge broker reader to the new file reader ( #14980 )
...
Currently, there are two sets of file readers in Doris, this pr rewrites the old broker reader with the new file reader.
TODO:
1. rewrite stream load pipe and kafka consumer pipe
2022-12-14 12:48:02 +08:00
f17b138cbd
[BugFix](regression) don't use sf1DataPath when stream load ( #15060 )
...
don't use sf1DataPath when stream load
2022-12-14 12:39:56 +08:00
30406ef580
[Chore](s2geo) add with gflags on s2geo ( #15059 )
...
add with gflags on s2geo
2022-12-14 12:19:01 +08:00
7854e9ae5a
[thirdparty](libgsasl) set include dir to enable to find gssapi.h ( #15037 )
...
If not set, it can't find gssapi.h, which is generated by krb5.
This will cause the libhdfs3 failed to visit hdfs with kerberos authentication
2022-12-14 12:07:13 +08:00
bbf3a5420d
[fix](statistics) fix missing scanBytes and scanRows in query statist… ( #14828 )
...
A patch for PR-14750. There's one modification missing in ISSUE-14750.
2022-12-14 09:37:05 +08:00
03e69d863e
[fix](nereids) Use precison and scale of decimal type column ( #15025 )
2022-12-13 23:39:04 +08:00
bc3a35d962
[typo](doc): modify the installation file ( #15036 )
2022-12-13 23:37:33 +08:00
271c28472a
[typo](docs)Fix doc ( #15051 )
2022-12-13 23:17:41 +08:00
04e5047674
[feature](Nereids): binder support bind join function. ( #15010 )
2022-12-13 21:31:44 +08:00
3d1be664b1
[feature](multi-catalog) support connecting to hive metastore with ke… ( #15026 )
...
Support kerberos authentication on hive external catalog
2022-12-13 16:48:46 +08:00
99c339d5cb
[refactor](regression) make nereids case in a separate dir ( #14990 )
...
add DateV2 and DateTimeV2 for Literal.uncheckCastTo()
move nereids tpch cases into suite nereids_tpch_p1
move nereids datav2 cases into suite nereids_datav2_p1
2022-12-13 16:28:41 +08:00
98ddb86ea2
[typo](doc)Update install-faq.md ( #15029 )
...
* [typo](doc) 1.2 set java home variable
2022-12-13 15:38:25 +08:00
284a3351f4
[Refactor](exec) refactor the code of datasink eos logic ( #15009 )
2022-12-13 15:33:08 +08:00
73ee352705
[fix](multi catalog)Fix convert_to_doris_type missing break for some cases ( #14992 )
2022-12-13 13:34:55 +08:00
e7a84e4a16
[fix](multi-catalog)fix page index thrift deserialize ( #15001 )
...
fix the err when parse page index: Couldn't deserialize thrift msg.
use two buffer to store column index and offset index msg, avoid parse them in a buffer
2022-12-13 13:33:19 +08:00
c767e0bc74
[fix](nereids) create select node for standalone PhysicalFilter node ( #14939 )
...
The PhysicalFilter can't be assigned to ExchangeNode, SortNode and UnionNode. The nereids would create a standalone SelectNode to do the filter work properly.
2022-12-13 13:29:45 +08:00
3caa9a19cc
[feature](Nereids) add binary & unary arithmetic expression ( #14867 )
...
binary arithmetic expression: div, ^, |, &, %
unary arithmetic expression: ~, +1
2022-12-13 12:58:40 +08:00
414566b56d
[feature](nereids) Support orderby and groupby int literal as ordinal of the select list expr ( #14862 )
2022-12-13 12:48:09 +08:00
d9fc46382b
[samples](java udf demo)Java udf demo aes ( #15011 )
...
* add java udf function demo
2022-12-13 12:09:03 +08:00
8fe0729835
[fix](multi catalog)Check orc file reader is not null before using it. ( #14988 )
...
The external table file path cache may out of date, which will cause orc reader to visit non-exist files.
In this case, orc file reader is nullptr.
This pr is to check the reader before using it to avoid core dump of visiting nullptr.
2022-12-13 11:27:51 +08:00
21676b8d81
[Fix](Table Valued function) fix the problem that can not catchthe exception thrown by the TableValuedFunctionRef constructor ( #14983 )
...
Put the generation of TableValuedFunctionIf in the analyze function, instead of the generation in the
TableValuedFunctionRef constructor.
2022-12-13 11:26:04 +08:00
5e26ba98bd
[enhancement](compaction) exist safely ( #15021 )
2022-12-13 10:57:27 +08:00
decabbb933
[Chore](s2geo) upgrade s2geo to 0.10.0 ( #15002 )
...
upgrade s2geo to 0.10.0
2022-12-13 10:34:51 +08:00
1200b22fd2
[function](round) compute accurate round value by decimal ( #14946 )
2022-12-13 09:53:43 +08:00
0d5291801d
[fix](load) fix that flush memtable concurrently may cause data inconsistency ( #15005 )
2022-12-13 09:27:35 +08:00
dcede52964
[typo](docs)add be config doris_scanner_row_bytes ( #15016 )
2022-12-13 09:25:28 +08:00
c25a7235f9
[Pipeline](load) support pipeline broker load ( #14940 )
...
support pipeline broker load
2022-12-13 00:28:36 +08:00
1f56279fd8
[Vectorized] Use SIMD to skip batches of null data in aggregation ( #10392 )
2022-12-12 23:40:31 +08:00
ad8ec809b0
[fix](nereids) FoldConstantRuleOnFe throw npe if function do not have child ( #15008 )
...
FoldConstantRuleOnFe missed one type of function, which do not have input arguments, like current_date
2022-12-12 22:48:21 +08:00
