Commit Graph

5948 Commits

Author SHA1 Message Date
8551ceaa1b [Bug][Vectorized] Fix use-after-free bug of memtable shrink (#11197)
Co-authored-by: lihaopeng <lihaopeng@baidu.com>
2022-07-26 16:10:44 +08:00
09224d7c74 [refactor] Rename Catalog to Env (#10702) 2022-07-26 15:49:08 +08:00
846716ac10 [feature](nereids): join reorder (#10479)
Enhance join reorder.

Add LAsscom (include with project).
Add Commute.
Add UT for join reorder
2022-07-26 15:35:00 +08:00
a3df2e5982 [enhance](*): improve code about optional (#11153)
improve code about optional
2022-07-26 15:32:42 +08:00
58795771f2 [feature](nereids) Add subquery expression and In expression and TPCH… (#11129)
1. Increase the expression of subquery and in.
2. Added tpch creation table and sql query, including original sql query and query rewritten by doris
3. Adjust the position of checkAnalyze
4. add exists subquery
2022-07-26 12:30:12 +08:00
8fc79e3ee4 rename aggregate properties (#11117)
1. Unify and refine property names in LogicalAggregate and PhysicalAggregate
2. Remove partitionExpressions in LogicalAggregate since it's a physical property, and should not appear in the logical plan. It should be generated when converting logical aggregate to physical aggregate or in enforcing rules.
2022-07-26 12:26:02 +08:00
e54b57107a [bug](FE): Fix show alter column return incorrect job progress (#11184)
bugfix:now show altr table return correct job progress
2022-07-26 11:49:14 +08:00
6a53cf8efa [dependency](be) Add vectorscan for support hypserscan on ARM (#11102)
hyperscan is a high-performance regular expression matching library, but can not be used on ARM.
vectorscan is an ARM port for hyperscan, it can be used as a drop in replacement.

Since hyperscan is original created by Intel and is popular and mature on x86, so we just use vectorscan
only for aarch64 when build thirdparty.
2022-07-26 11:28:25 +08:00
3e3b2d15d4 [bug]string pad functions should always be nullable (#11140)
* string pad functions should always be nullable
2022-07-26 10:20:11 +08:00
1788e2fe6c [fix](fe-ut) Ignore invalid datetimev2 partition value in UT (#11178) 2022-07-26 09:00:38 +08:00
9ec8e732ae [regression-test]Add order-by to test_bitmap_function (#11174) 2022-07-25 18:24:40 +08:00
bbe08b34ba [Bug](be-ut) Fix the timezone dependency in UT (#11148) 2022-07-25 18:15:05 +08:00
37dff975a7 [bugfix] fix ASAN error alloc-dealloc-mismatch (#11168) 2022-07-25 18:14:20 +08:00
a9fc078b55 [RegressionTest][Array] Fix the bug of regression of array test: (#11173)
1. [] do not have a proper array nested type, cause BE coredump
2. [abc] or ['abc'] load by vectorized load get error result

Co-authored-by: lihaopeng <lihaopeng@baidu.com>
2022-07-25 18:13:49 +08:00
00e2944102 [bugfix] fix coredump caused by wrong type cast of OlapScanNode (#11165) 2022-07-25 17:57:53 +08:00
0d50a658f5 [fix](config) Fix uninitialized config validator (#11022)
* [fix](config) Fix uninitialized config validator

If we don't set any validator, the original implementation will cause
seg-fault due to nullptr of `RegisterConfValidator::_s_field_validator`,
which initial value relies on the first config validator definition.

We need to skip finding validator from
uninitialized `RegisterConfValidator::_s_field_validator`.
2022-07-25 15:10:55 +08:00
c641de4629 [fix](fe-ut) Fix FE UT failures (#11155) 2022-07-25 14:09:20 +08:00
93cb80c9cb [test] use suffix of directory as group name and use directory as dbname (#11142)
* use suffix of directory as group name and use directory as dbname

We can rename tpcds_sf1 to tpcds_sf1_p1, then tpcds_sf1 will be in group
p1.  We will group cases to p0, p1, p2, p3 in the future.

p0: function cases running in seconds.
p1: cases with expected out running in minutes, like tpcds_sf1
p2: cases with expected out running in hours, like tpcds_sf10 tpcds_sf100
p3: cases without without expected out to test core dump.
2022-07-25 12:10:31 +08:00
54f878b781 [feature-wip](multi-catalog) Support orc format file split for file scan node (#11046) 2022-07-25 11:41:46 +08:00
d2457ce8f9 [Feature](Nereids) Add an annotation @Developing (#11077)
Currently, the new optimizer is under development. We want to merge not fully developed features into the code, but not be used in the main code path.

Added an annotation @Developing to mark these features.
2022-07-25 11:24:26 +08:00
84ce7eddf6 [feature-wip](parquet-reader) add thrift file for new parquet reader (#11150) 2022-07-25 10:11:15 +08:00
d9066440f6 [fix](fe_plugins) update fe plugins' dependency to fe 1.0-SNAPSHOT (#11141) 2022-07-25 09:13:15 +08:00
829d534e12 [Improvement] Replace switch with constexpr to boost date functions (#11134) 2022-07-23 22:58:59 +08:00
573ca0bbc2 FIX: odbc string error (#11096) 2022-07-23 18:19:11 +08:00
Pxl
6e98ebba27 [Vectorized] Support sort combinator (#10469) 2022-07-23 17:58:31 +08:00
baef77a8f0 [feature](nereids): add equals for expression (#11067)
add equals for datatype and some expression.

Fix some small problem like arrangement.
2022-07-23 17:32:06 +08:00
babab5d535 [feature-wip] support datetimev2 (#11085) 2022-07-23 16:07:59 +08:00
0b6d2ae290 [fix] Move s3 fs connect outside the lock critical area (#11026)
* fix potential bug of S3FileSystem

* move s3 fs connect outside the lock critical area
2022-07-23 16:06:29 +08:00
83eb5a0e44 [enhancement] Optimize the log content to make the meaning clearer (#11133)
* [enhancement] Optimize the log content to make t
2022-07-23 12:59:25 +08:00
7c7852994c (fix)(Nereids) fix ssb and add regression test case (#11095)
current nereids planner execute ssb will run into dead loop and crash be, this pr fix this problem and add some regression test case prevent execute ssb failed
2022-07-23 12:41:47 +08:00
fdb4193e1b [Vectorized][Refactor] Refactor the function of tuple_is_null, only do work in hash join node (#11109) 2022-07-23 11:50:07 +08:00
6422a5d4f7 [improvement](arrow) add arrow block convertion time profile (#11072)
* [improvement](arrow) add arrow block convertion time profile
2022-07-22 22:11:33 +08:00
98abb8bc1f fix empty storage policy, be refresh exception log. (#11123)
* fix empty storage policy, be refresh exception log.

* fix log level
2022-07-22 22:10:16 +08:00
b7c9007776 [improvement][agg]Process aggregated results in the vectorized way (#11084) 2022-07-22 22:04:43 +08:00
ad31b6c902 [bugfix and improvement]fix mem tracker for load and simplify some macros (#11125) 2022-07-22 21:59:36 +08:00
65dd8eb885 Update init-env.sh (#11111)
This script is missing "!"
2022-07-22 21:55:12 +08:00
aa1bcdbc18 [Bug] Show create table null pointer of storage policy and error htttp path of tablet info (#10950)
Co-authored-by: lihaopeng <lihaopeng@baidu.com>
2022-07-22 20:55:35 +08:00
49a17bea99 [regression]add the cases for csv/orc/parquet file format (#11082)
Co-authored-by: hucheng01 <hucheng01@baidu.com>
2022-07-22 20:54:34 +08:00
d17c906eb7 [chore](FE)add license header check in fe's checkstyle (#11076)
Add license header check in fe's checkstyle
2022-07-22 18:37:32 +08:00
34f328aa57 [feature] (Nereids) Merge memo group recursively (#11043)
In Memo.copyIn( plan, group1, isRewrite), one branch is that the plan is already recorded in Memo, and owned by group 'group2'. In such case, 'group1' should be merged with 'group2', because they are equivalent.
After merge, the upper level of 'group1', saying 'p1 = group1.getLogicalExpression().getOwnerGroup()' of 'group1', and that of 'group2', saying 'p2', are equivalent. We need to merge 'p1' and 'p2'. And this process is recursive.
2022-07-22 18:31:32 +08:00
0681e4f04f [Refactor](Nereids) Remove expression type. (#11066)
ExpressionType is duplicated with Java class type info, so removed it.
2022-07-22 17:48:18 +08:00
6963c41a04 [dependency] Upgrade Apache Commons Validator version to the latest one (#10508) 2022-07-22 17:03:46 +08:00
764abfe72f [enhancement](community): add ci for close outdated PR. (#11088) 2022-07-22 17:00:17 +08:00
3744321f01 [feature-wip](array-type) add function array_union/array_except/array_intersect (#10781)
Add array_union/array_except/array_intersect function.
2022-07-22 13:50:13 +08:00
9d21b2154d [Fix](Array) correct the offset when using get_data_at from _item_convertor (#11094)
get_data_at should use offset - offsets[start_index] since
start_index may be changed after OlapColumnDataConvertorArray::set_source_column.
Using just offset may access the memory out of _item_convertor's data range,
2022-07-22 11:25:17 +08:00
4003489bd0 [fix](update) check LOAD priv for update stmt (#11099) 2022-07-22 11:24:44 +08:00
40c8853c5d [Fix] Fix select external table return “Lost connection to MySQL server during query” error 2022-07-22 11:24:09 +08:00
552c0568fe [sample]Add flink doris connector 1.1 sample code (#10970)
* Add flink doris connector 1.1 sample code
2022-07-22 10:18:27 +08:00
900430f55e [docs] update invaid links (#10272) 2022-07-22 09:59:53 +08:00
9017afb7a8 [enhancement](workflow) Add a workflow for Clang check (#11083)
* Add a workflow for Clang check
2022-07-22 09:14:21 +08:00