Commit Graph

13 Commits

Author SHA1 Message Date
Pxl
2ccb69dbed [Feature](materialized-view) support some case unmached to materialized-view (#30036)
same column appears in key and value like select id,count(id) group by id;
complex expr in sum select sum(if(xxx));
2024-01-18 12:03:07 +08:00
8c05f7a784 [refactor](cluster)(step-4) remove cluster related to Database (#27861)
Issue Number: #19897

Remove `default_cluster` prefix related to database.
When upgrading, all prefix will be removed.
2023-12-16 18:28:53 +08:00
f1d90ffc4e [regression](nereids) add test case for partition prune (#26849)
* list selected partition name in explain
* add prune partition test case (multi-range key)
2023-11-14 11:51:32 +08:00
e9554e36a8 [fix](nereids)disable parallel scan in some case (#25089) 2023-10-11 16:32:09 +08:00
fc12362a6d [feature-wip](arrow-flight)(step2) FE support Arrow Flight server (#24314)
This is a POC, the design documentation will be updated soon
2023-09-20 14:42:54 +08:00
ae8a26335c [opt](hive)opt select count(*) stmt push down agg on parquet in hive . (#22115)
Optimization "select count(*) from table" stmtement , push down "count" type to BE.
support file type : parquet ,orc in hive .

1. 4kfiles , 60kwline num 
    before:  1 min 37.70 sec 
    after:   50.18 sec

2. 50files , 60kwline num
    before: 1.12 sec
    after: 0.82 sec
2023-07-29 00:31:01 +08:00
46ce97a190 [enhance](planner)convert 'or' into 'in-predicate' (#15737)
in previous [PR 12872](https://github.com/apache/doris/pull/12872), we convert multi equals on same slot into `in predicate`. for example, `a =1 or a = 2` => `a in (1, 2)`

This pr makes 4 changes about convert or to in:
1. fix a bug: `Not IN`  is merged with equal. `a =1 or a not in (2, 3)` => `a in (1, 2, 3)`
2. extends this rule on more cases
  - merge for more than one slot: 'a =1 or a = 2 or b = 3 or b = 4' => `a in (1, 2) or b in (3, 4)`
  - merge skip not-equal and not-in: 'a =1 or a = 2 or b !=3 or c not in (1, 2)' => 'a in (1, 2) or b!=3 or c not in (1,2)`
3. rewrite recursively. 
4. OrToIn is implemented in ExtractCommonFactorsRule. This rule will generate new exprs. OrToIn should apply on such generated exprs. for example `(a=1 and b=2) or (a=3 and b=4)` => `(a=1 or a=3) and (b=2 or b=4) and [(a=1 and b=2) or (a=3 and b=4)]` => `a in (1,3) and b in (2 ,4) and [(a=1 and b=2) or (a=3 and b=4)]` 

In addition, this pr add toString() for some Expr.
2023-01-18 12:33:20 +08:00
25b257e37c [enhancement](session var) varariable to control whether to rewrite OR to IN or not (#15437) 2022-12-29 14:50:32 +08:00
0550dfaeb2 [enhancement](rewrite) add OrToIn rule and fix ExtractCommonFactorsRule apply problems (#12872)
Co-authored-by: wuhangze <wuhangze@jd.com>
2022-12-27 18:39:53 +08:00
56f7ba19c0 [opt](planner) add session var: COMPACT_EQUAL_TO_IN_PREDICATE_THRESHOLD (#15225)
in previous pr(#14876) we compact equals like "a=1 or a=2 or a = 3 " in to "a in (1, 2, 3)"
this pr set a lower bound for the number of equals COMPACT_EQUAL_TO_IN_PREDICATE_THRESHOLD (default is 2)

for performance reason, we create a hashSet to collect literals, like {1,2,3}. and hence, the literals in "in-predicates" are in random order.

for regression test, if we need stable output of explain string, set COMPACT_EQUAL_TO_IN_PREDICATE_THRESHOLD to a large number to avoid compact rule.
2022-12-21 21:10:47 +08:00
320b264c9d [feature](planner) compact multi-euqals to in-predicate #14876 2022-12-20 09:43:34 +08:00
efe813ba60 [fix](test)(explain) add full qualified name for scan node explain string (#13777)
1.
In the "explain" result of SQL, the table name in `ScanNode` should be full qualified with dbname.
And for olap scan node, the selected index name should not be "null".

2.
Remove `tpch_sf1_p1/tpch_sf1/nereids/` in regression test, it will be fixed later.
2022-10-30 13:24:48 +08:00
ee4d9d4347 [improvement](test) group some cases and group a case to p0 if it is not grouped (#11548) 2022-08-06 15:12:08 +08:00