Commit Graph

5461 Commits

Author SHA1 Message Date
166b4a18ae [fix](image) fix bug that latestValidatedImageSeq may not be the second largest image id (#11201)
* [fix](image) fix bug that latestValidatedImageSeq may not be the second largest image id

When traversing the image files in the meta directory,
it cannot be guaranteed to be traversed in the order of imageid size

For example, if it traverses the image file with orders like: 3,5,4,1,
then latestImageSeq is 5, but latestValidatedImageSeq is 3, which is wrong.
2022-07-26 22:35:31 +08:00
8a6c9d870e [enhancement](nereids) Fix the wrong order of arguments of Assert.assertEquals (#11207) 2022-07-26 20:33:32 +08:00
2e210fb363 [bugfix]fix constant argument coredump (#11200) 2022-07-26 19:30:06 +08:00
fcdb543e17 [refactor] (Nereids) add equals for OrderKey (#11192)
LogicalSort.equals() method depends on OrderKey.equals(), which is not defined correctly.
This pr defines OrderKey.equals() to enable correctly comparing LogicalSort.
2022-07-26 18:55:27 +08:00
e99f617b21 [Refactor](Nereids) Move analysis logic to dedicated class NereidsAnalyzer. (#11171)
This PR proposes to move the analysis logic to the dedicated class NereidsAnalyzer, which has the following benefits:

Unify the analysis logic in production and test files.
Facilitate analyzing subquery plans within different scopes.
2022-07-26 18:32:07 +08:00
f5479fa80e [Sample] Add Flink Connector Sample Code (#11120)
Co-authored-by: wudi <>
2022-07-26 18:00:51 +08:00
03d466febe [UDAF](sample) impl rpc avg udaf (#11132) 2022-07-26 18:00:13 +08:00
a1ad978960 [Bug]Fix select command denied for user for specified table 2022-07-26 17:59:28 +08:00
2c34b6cf2b optimization the result check for test_json_load (#11160)
Co-authored-by: hucheng01 <hucheng01@baidu.com>
2022-07-26 17:58:30 +08:00
0f6ec629ab [Bug](fe-ut) Fix UT when test cases in package (#11177) 2022-07-26 17:57:42 +08:00
f46a801b1b [FIX]string pad function should be always nullable for both string and varchar type (#11196) 2022-07-26 17:55:06 +08:00
61fb39a5ca [Enhancement](Nereids) add some basic data structure definitions. (#10981)
add decimal 、date、datetime type
add DateLiteral
add Interval for timestamp calculation
2022-07-26 17:51:59 +08:00
823088a9eb [FOLLOW-UP] (datetimev2) complete date function ut and built-in function declaration (#11154) 2022-07-26 17:48:57 +08:00
8551ceaa1b [Bug][Vectorized] Fix use-after-free bug of memtable shrink (#11197)
Co-authored-by: lihaopeng <lihaopeng@baidu.com>
2022-07-26 16:10:44 +08:00
09224d7c74 [refactor] Rename Catalog to Env (#10702) 2022-07-26 15:49:08 +08:00
846716ac10 [feature](nereids): join reorder (#10479)
Enhance join reorder.

Add LAsscom (include with project).
Add Commute.
Add UT for join reorder
2022-07-26 15:35:00 +08:00
a3df2e5982 [enhance](*): improve code about optional (#11153)
improve code about optional
2022-07-26 15:32:42 +08:00
58795771f2 [feature](nereids) Add subquery expression and In expression and TPCH… (#11129)
1. Increase the expression of subquery and in.
2. Added tpch creation table and sql query, including original sql query and query rewritten by doris
3. Adjust the position of checkAnalyze
4. add exists subquery
2022-07-26 12:30:12 +08:00
8fc79e3ee4 rename aggregate properties (#11117)
1. Unify and refine property names in LogicalAggregate and PhysicalAggregate
2. Remove partitionExpressions in LogicalAggregate since it's a physical property, and should not appear in the logical plan. It should be generated when converting logical aggregate to physical aggregate or in enforcing rules.
2022-07-26 12:26:02 +08:00
e54b57107a [bug](FE): Fix show alter column return incorrect job progress (#11184)
bugfix:now show altr table return correct job progress
2022-07-26 11:49:14 +08:00
6a53cf8efa [dependency](be) Add vectorscan for support hypserscan on ARM (#11102)
hyperscan is a high-performance regular expression matching library, but can not be used on ARM.
vectorscan is an ARM port for hyperscan, it can be used as a drop in replacement.

Since hyperscan is original created by Intel and is popular and mature on x86, so we just use vectorscan
only for aarch64 when build thirdparty.
2022-07-26 11:28:25 +08:00
3e3b2d15d4 [bug]string pad functions should always be nullable (#11140)
* string pad functions should always be nullable
2022-07-26 10:20:11 +08:00
1788e2fe6c [fix](fe-ut) Ignore invalid datetimev2 partition value in UT (#11178) 2022-07-26 09:00:38 +08:00
9ec8e732ae [regression-test]Add order-by to test_bitmap_function (#11174) 2022-07-25 18:24:40 +08:00
bbe08b34ba [Bug](be-ut) Fix the timezone dependency in UT (#11148) 2022-07-25 18:15:05 +08:00
37dff975a7 [bugfix] fix ASAN error alloc-dealloc-mismatch (#11168) 2022-07-25 18:14:20 +08:00
a9fc078b55 [RegressionTest][Array] Fix the bug of regression of array test: (#11173)
1. [] do not have a proper array nested type, cause BE coredump
2. [abc] or ['abc'] load by vectorized load get error result

Co-authored-by: lihaopeng <lihaopeng@baidu.com>
2022-07-25 18:13:49 +08:00
00e2944102 [bugfix] fix coredump caused by wrong type cast of OlapScanNode (#11165) 2022-07-25 17:57:53 +08:00
0d50a658f5 [fix](config) Fix uninitialized config validator (#11022)
* [fix](config) Fix uninitialized config validator

If we don't set any validator, the original implementation will cause
seg-fault due to nullptr of `RegisterConfValidator::_s_field_validator`,
which initial value relies on the first config validator definition.

We need to skip finding validator from
uninitialized `RegisterConfValidator::_s_field_validator`.
2022-07-25 15:10:55 +08:00
c641de4629 [fix](fe-ut) Fix FE UT failures (#11155) 2022-07-25 14:09:20 +08:00
93cb80c9cb [test] use suffix of directory as group name and use directory as dbname (#11142)
* use suffix of directory as group name and use directory as dbname

We can rename tpcds_sf1 to tpcds_sf1_p1, then tpcds_sf1 will be in group
p1.  We will group cases to p0, p1, p2, p3 in the future.

p0: function cases running in seconds.
p1: cases with expected out running in minutes, like tpcds_sf1
p2: cases with expected out running in hours, like tpcds_sf10 tpcds_sf100
p3: cases without without expected out to test core dump.
2022-07-25 12:10:31 +08:00
54f878b781 [feature-wip](multi-catalog) Support orc format file split for file scan node (#11046) 2022-07-25 11:41:46 +08:00
d2457ce8f9 [Feature](Nereids) Add an annotation @Developing (#11077)
Currently, the new optimizer is under development. We want to merge not fully developed features into the code, but not be used in the main code path.

Added an annotation @Developing to mark these features.
2022-07-25 11:24:26 +08:00
84ce7eddf6 [feature-wip](parquet-reader) add thrift file for new parquet reader (#11150) 2022-07-25 10:11:15 +08:00
d9066440f6 [fix](fe_plugins) update fe plugins' dependency to fe 1.0-SNAPSHOT (#11141) 2022-07-25 09:13:15 +08:00
829d534e12 [Improvement] Replace switch with constexpr to boost date functions (#11134) 2022-07-23 22:58:59 +08:00
573ca0bbc2 FIX: odbc string error (#11096) 2022-07-23 18:19:11 +08:00
Pxl
6e98ebba27 [Vectorized] Support sort combinator (#10469) 2022-07-23 17:58:31 +08:00
baef77a8f0 [feature](nereids): add equals for expression (#11067)
add equals for datatype and some expression.

Fix some small problem like arrangement.
2022-07-23 17:32:06 +08:00
babab5d535 [feature-wip] support datetimev2 (#11085) 2022-07-23 16:07:59 +08:00
0b6d2ae290 [fix] Move s3 fs connect outside the lock critical area (#11026)
* fix potential bug of S3FileSystem

* move s3 fs connect outside the lock critical area
2022-07-23 16:06:29 +08:00
83eb5a0e44 [enhancement] Optimize the log content to make the meaning clearer (#11133)
* [enhancement] Optimize the log content to make t
2022-07-23 12:59:25 +08:00
7c7852994c (fix)(Nereids) fix ssb and add regression test case (#11095)
current nereids planner execute ssb will run into dead loop and crash be, this pr fix this problem and add some regression test case prevent execute ssb failed
2022-07-23 12:41:47 +08:00
fdb4193e1b [Vectorized][Refactor] Refactor the function of tuple_is_null, only do work in hash join node (#11109) 2022-07-23 11:50:07 +08:00
6422a5d4f7 [improvement](arrow) add arrow block convertion time profile (#11072)
* [improvement](arrow) add arrow block convertion time profile
2022-07-22 22:11:33 +08:00
98abb8bc1f fix empty storage policy, be refresh exception log. (#11123)
* fix empty storage policy, be refresh exception log.

* fix log level
2022-07-22 22:10:16 +08:00
b7c9007776 [improvement][agg]Process aggregated results in the vectorized way (#11084) 2022-07-22 22:04:43 +08:00
ad31b6c902 [bugfix and improvement]fix mem tracker for load and simplify some macros (#11125) 2022-07-22 21:59:36 +08:00
65dd8eb885 Update init-env.sh (#11111)
This script is missing "!"
2022-07-22 21:55:12 +08:00
aa1bcdbc18 [Bug] Show create table null pointer of storage policy and error htttp path of tablet info (#10950)
Co-authored-by: lihaopeng <lihaopeng@baidu.com>
2022-07-22 20:55:35 +08:00