Commit Graph

4796 Commits

Author SHA1 Message Date
f49284036e [Enhancement] Refactor functions in int_exp by templates (#9939) 2022-06-04 11:53:31 +08:00
da33a48f39 [refactor](policy) Refactor the hierarchy of Policy. (#9786)
The RowPolicy extends Policy
2022-06-04 11:29:09 +08:00
3031919e8f [fix] (planner) slot nullable does not set correctly when plan outer join with inline view (#9927)
- set inline view's slot descriptor to nullable in register column ref
- propagate slot nullable when generate inline view's query node in SingleNodePlanner
2022-06-03 17:50:10 +08:00
937491098e [fix] fix grammar of ADMIN SHOW TABLET STORAGE FORMAT stmt (#9938) 2022-06-03 17:49:34 +08:00
c996334ad1 [improvement] Optimize send fragment logic to reduce send fragment timeout error (#9720)
This CL mainly changes:
1. Reducing the rpc timeout problem caused by rpc waiting for the worker thread of brpc.
    1. Merge multiple fragment instances on the same BE to send requests to reduce the number of send fragment rpcs
    2. If fragments size >= 3, use 2 phase RPC: one is to send all fragments, two is to start these fragments. So that there
         will be at most 2 RPC for each query on one BE.

3. Set the timeout of send fragment rpc to the query timeout to ensure the consistency of users' expectation of query timeout period.

4. Do not close the connection anymore when rpc timeout occurs.
5. Change some log level from info to debug to simplify the fe.log content.

NOTICE:
1. Change the definition of execPlanFragment rpc, must first upgrade BE.
3. Remove FE config `remote_fragment_exec_timeout_ms`
2022-06-03 15:47:40 +08:00
cacad64d2c [fix] Error when compiling under ARM architecture machine, fixed a loop (#9948) 2022-06-03 08:00:55 +08:00
67fa1fcf2a [fix] fix invalid SQL rewrite for field in materialized view (#9877) 2022-06-02 23:43:13 +08:00
3241cc2bfd [doc][fix]English document title case (#9935) 2022-06-02 20:52:58 +08:00
Pxl
c0ad1be1bd [Enhancement][Chore] remove breakpad and unused variable (#9937) 2022-06-02 20:52:17 +08:00
c426c2e4b1 [Vectorized-Load] Support vectorized load table with materialized view (#9923)
* [Vectorized-Load] Support vectorized load table with materialized view

* fix ut

Co-authored-by: lihaopeng <lihaopeng@baidu.com>
2022-06-02 14:59:01 +08:00
df3406da30 [test]add string sql functions test case (#9904)
* add string sql functions test case

* add like and regexp function case vectorized true

* Cancellation of Chinese language Test

* Cancel the Chinese test for the second time
2022-06-02 14:53:07 +08:00
0be1b0d242 [regressiontest]add json function test case (#9917) 2022-06-02 14:52:39 +08:00
7792aebd21 [test]add conditional function test case (#9913)
* add conditional function test case

* Update test_conditional_function.groovy

Delete useless code

* optimize the code
2022-06-02 14:52:03 +08:00
2ad38e6ab2 [regressiontest]add hash function test case (#9918) 2022-06-02 14:51:25 +08:00
f681c0891f [regression test]add encryption function test case (#9920) 2022-06-02 14:47:32 +08:00
dcf18ac322 [fix](hive) fix bug of invalid user info in external table's scan node (#9908)
Fix the hive external table scan node null exception
Now hive external table query will fail when use local user@ip
2022-06-02 10:41:40 +08:00
4ea5782838 [fix](Function) fix to_bitmap to return always not nullable (#9859) 2022-06-02 10:37:45 +08:00
9f26c6a34f [chore] fix some compatible on old linux distributions (#9815) 2022-06-02 10:37:10 +08:00
e896fffd76 [Vectorized][Function] fix bitmap_intersect get wrong result (#9907) 2022-06-01 23:51:52 +08:00
fccb7b8055 [fix](planner) Fix the bug of can't query the data of new added partition when set partition_prune_algorithm_version = 2 (#9844) 2022-06-01 23:44:14 +08:00
2082d0a01f [optimize](planner)remove redundant conjuncts on plan node (#9819) 2022-06-01 23:43:08 +08:00
2fc2113b05 [feature] Support show proc BackendLoadStatistic (#9618)
The proc info method already exists in `ClusterLoadStatistic.getBackendStatistic`, I'll add a proc node to show it.
2022-06-01 23:30:10 +08:00
47dfdd8e09 [fix](storage) Disable compaction before schema change is actually executed(#9032) (#9065)
As in issue, the combination and schema change at the same time may lead to version intersection.
Describe the overview of changes.
1. Do not do compaction before schema change is actually executed.
2. Set tablet as bad when it has version intersection.
3. Do not do schema change when it can not find appropriate versions to delete in new tablet.
4. Do not change rowsets after compaction if the rowsets of the tablet has changed.
2022-06-01 23:29:18 +08:00
5056a1b2b6 [fix][doc]Fix the usage of bitmap on unique model (#9903) 2022-06-01 22:38:17 +08:00
8effdd95a7 [fix](routine-load) fix bug that routine load task can not find backend (#9902)
Introduced from #9492.
2022-06-01 17:55:30 +08:00
92babc7a47 [improvement][fix](planner) Add a rewrite rule to optimize InPredicate. (#9739)
1. Convert child expressions in InPredicate to column type and discard child expressions in them that cannot be converted exactly.
2. Fix the bug of ColumnRange exception caused by InPredicate child expressions type conversion.
3. Fix the problem that the tablet could not be hit due caused by InPredicate child expressions type conversion.
2022-06-01 15:26:32 +08:00
0b7c4ca06e [fix][doc]Version correction (#9899)
* [doc]Add Doris join optimization documentation

Add Doris join optimization documentation

* Revert "[doc]Add Doris join optimization documentation"

This reverts commit 418ca1dfb0b91cf45d021c6d841e16e48432fcc3.

* Version correction

Version correction
2022-06-01 15:22:15 +08:00
Pxl
ac08c7ac91 [fix](vectorized) fix vcast expr input wrong row number (#9520) 2022-06-01 15:19:31 +08:00
4ab7694a7f [Enhancement](Nereids)rewrite framework used in Memo (#9807)
Issue Number: close #9627 , #9628

This PR introduce two essentials for Nereids

1. pattern match iterator used in memo

pattern match iterator is implemented by two iterators nested within each other: GroupExpressionIterator and GroupIterator.
GroupExpressionIterator use GroupIterator to get all children Plan which matching pattern and use them as children to generate pattern matched plan.
GroupIterator use GroupExpressionIterator to get all pattern matched Plan related to GroupExpressions in itself.

2. plan rewrite framework for memo

Rewrite framework is implemented by two jobs: RewriteTopDownJob and RewriteBottomUpJob
Both of them takes a group, a set of rules that need to be applied, and a context as construction parameters.
RewriteTopDownJob apply these jobs from top to down one by one.
RewriteBottomUpJob apply these jobs from bottom to up one by one.
When one rule rewrites plan tree at a plan node. This plan node will be applied all rules again until no rules can rewrite it.
2022-06-01 15:12:58 +08:00
632f7a3d3d [Feature] add weekday function on vectorized engine (#9901) 2022-06-01 14:47:37 +08:00
827b948d69 [test] Add datetime function test for regression test. (#9897)
* Modify export example in en and zh-cn doc.

* Add datetime function test for regression test.

* Add some invalid date to convert_tz

Co-authored-by: smallhibiscus <844981280>
2022-06-01 12:31:22 +08:00
03a1eec3a5 [doc][fix]Update the description of the doris In_memory attribute to avoid users from interpreting it as an in-memory table (#9892)
Update the description of the doris In_memory attribute to avoid users from interpreting it as an in-memory table
2022-06-01 08:47:56 +08:00
2e0d05e0dc [test]add gis function test case (#9888) 2022-06-01 08:12:34 +08:00
00719db3a2 [bugfix]handle ComlumnDictionary in evaluate_and and evaluate_or (#9818)
* handle ColumnDictory in evaluate_or

We need to handle ComlumnDictory in evaluate_or, otherwise delete handler
would triger a core dump.

* handle ComlumnDictionary in evaluate_and

Because there is only one difference between evaluate_and and
evaluate_or, that is or and delete, I merge two macros into one.

Delete handlers also trigger evaluate_and, i am not sure if column
dictionary would be used in evaluate_and.

* clang format

* fix short circut for evaluate_and and evaluate_or

* clang format
2022-06-01 08:10:43 +08:00
7498233601 [tools][doc]add ssb flat tools, and update performance doc (#9827) 2022-06-01 08:09:55 +08:00
7b65995400 [test] add rollup test for regression-test (#9832) 2022-06-01 08:08:56 +08:00
ec546e287b [regressiontest]Adding Window Funnel function causes be crash for regression test (#9889) 2022-06-01 08:06:25 +08:00
35f99faa0a [Bug][Vectorized] fix core dump on vcase_expr::close (#9893)
Co-authored-by: lihaopeng <lihaopeng@baidu.com>
2022-06-01 08:05:09 +08:00
0376ca17f3 [Enhancement] Remove minidump (#9894) 2022-06-01 08:04:24 +08:00
f3193c5ea3 [improvement]opt column_dictinary range filter (#9881)
* opt column_dictinary range filter

* fomart
2022-05-31 22:30:05 +08:00
54e9d49718 [Bug][Vectorized] Fix call nvl function core dump (#9883)
Co-authored-by: lihaopeng <lihaopeng@baidu.com>
2022-05-31 22:18:38 +08:00
Pxl
d34d631519 [bugfix]fix TableFunctionNode memory leak (#9853) 2022-05-31 19:20:22 +08:00
ec5027a00a [regressiontest]add bitmap index test case (#9880) 2022-05-31 19:18:43 +08:00
3025a23f83 [test]Add MATERIALIZED VIEW test for regression test (#9855)
* add MATERIALIZED VIEW test

* Fix DESC message does not match out file

* fix
2022-05-31 19:17:56 +08:00
c8d303a82c [bugfix] Fix BE core about vectorized join build thread memtracker switch, and FileStat duplicate 2022-05-31 19:12:42 +08:00
Pxl
fa50b63cee fix core dump on vcase_expr::close (#9875) 2022-05-31 15:45:39 +08:00
0cba6b7d95 [Bug][Fix] One Rowset have same key output in unique table (#9858)
Co-authored-by: lihaopeng <lihaopeng@baidu.com>
2022-05-31 12:29:16 +08:00
7199102d7c [Opt][VecLoad] Opt the vec stream load performance (#9772)
Co-authored-by: lihaopeng <lihaopeng@baidu.com>
2022-05-31 11:53:32 +08:00
e2b93a4165 [feature-wip](multi-catalog) Add basic class and interface for multi catalog support 2022-05-31 11:32:44 +08:00
f9d969cd10 [doc] fix alter resource doc style (#9866) 2022-05-31 09:53:46 +08:00