Commit Graph

18429 Commits

Author SHA1 Message Date
ae8e6fbe59 [Fix](inverted index) fix empty array index writer bug (#25984) 2023-10-27 11:27:44 +08:00
cedab51676 [enhancement](UT) add unit test cases about bitmap (#25867)
* [fix](bitmap) incorrect result of operator ==

* [enhancement](UT) add unit test cases about bitmap
2023-10-27 11:27:14 +08:00
6a62835f7e [fix](docker) Add docker ignore files and change docker default sourc… (#25696) 2023-10-27 11:23:21 +08:00
a3ac8b98b8 [fix](planner)nvl should match function signature in the same way as ifnull (#25849) 2023-10-27 11:13:30 +08:00
8206252e0a [Performance](Nereids): refactor GroupMatching (#25960)
Refactor GroupMatching to improve performance.
2023-10-27 10:56:42 +08:00
bac523956f [Update](clucene) update clucene version to latest (#25981) 2023-10-27 10:42:56 +08:00
0eb86919cc [fix](block-convertor) clear filter bitmap per batch to get the correct filtered rows (#25989) 2023-10-27 10:40:07 +08:00
7a69bdd1cd [regression](fe) Add more regression test for FE (#25976) 2023-10-27 10:27:22 +08:00
d0eaf95977 [cases](regression-test) Add alter table properties and alter/rename view test (#25971) 2023-10-27 10:27:06 +08:00
fa06c7517a [feature](meta) Add thrift rpc to get db/table/backend meta (#25943)
Signed-off-by: Jack Drogon <jack.xsuperman@gmail.com>
2023-10-27 10:12:58 +08:00
b32557e195 [fix](stats) Fix potential NPE when doing auto analyze (#25908) 2023-10-26 20:47:57 -05:00
f737ba815d [fix](nereids) push down subquery exprs in non-distinct agg functions (#25955) 2023-10-26 20:36:10 -05:00
c86fad7cbd [Fix](orc-reader) Fix orc decimal128 scale issue. (#25977) 2023-10-26 08:50:18 -05:00
d9a3aa5c06 [pipelineX](improvement) improve local shuffle (#25964) 2023-10-26 21:42:43 +08:00
36be7dcc13 [enhancement](regression) add normal query before date convert for debug (#25970)
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
2023-10-26 21:37:28 +08:00
69015df767 [Fix](inverted index) reorder ConjunctionQuery deconstruct order (#25972) 2023-10-26 08:34:09 -05:00
1e39636fbf [fix](nereids) fix wrong result precision for add/sub (#25751) 2023-10-26 08:28:34 -05:00
9faa6e083b [fix](udf) avoid exception when fail to find udf in replay logic (#25965)
When replaying drop function edit log, the function may not be found, causing runtime exception and
FE will fail to start.
The function SHOULD be exist, but the reason is still unknown.
I change the logic to NOT throw exception if function is not found.
This is a workaround to make sure FE can start, and add some log for later debug.
2023-10-26 08:19:13 -05:00
a2a157fcde [ci](pipeline) add required of "P0 Regression PipelineX (Doris Regression)" (#25967) 2023-10-26 19:17:12 +08:00
678dc366e0 [fix](export) fix timeout property not work for export job (#25913)
Co-authored-by: caiconghui1 <caiconghui1@jd.com>
2023-10-26 18:51:57 +08:00
2679fa4ea7 [improvement](tablet clone) furthur repair replicas should be check even if they are versions catchup (#25551) 2023-10-26 18:14:40 +08:00
c3527672a5 [refactor & pipelineX][pick fix] Pick fix of predicate pushdown to pipelineX (#25953)
Co-authored-by: JackDrogon <jack.xsuperman@gmail.com>
2023-10-26 18:04:43 +08:00
c1d64a7128 [Feature](datatype) Add IPv4/v6 data type for doris (#24965) 2023-10-26 17:33:28 +08:00
9705ec760c [fix](regression-test) use correct dataset for unique_with_mow_p2 (#25653) 2023-10-26 16:54:27 +08:00
642c701819 [doc](partial update) add flink-connector docs (#25843) 2023-10-26 16:53:06 +08:00
de06a2f8b7 [minor](test) remove useless date case (#25941)
remove case
```sql
select datediff('10000-10-31', '2010-10-15')
```

because Nereids throw exception when cast '10000-10-31' to date,
but legacy planner return null on it
2023-10-26 03:16:11 -05:00
1ba8a9bae4 [feature-wip](executor)Fe send topic info to be (#25798) 2023-10-26 15:52:48 +08:00
2229d82acd [fix](Nereids) should not put bound expr into unbound group by list (#25938)
we put bound expr into unbound group by list by mistake.
This will lead to bind twice on some exprssion.
Since binding is not idempotent, below exception will be thrown for sql

```sql
select k5 / k5 as nu, sum(k1) from test group by nu order by nu nulls first
```

```
Caused by: org.apache.doris.nereids.exceptions.AnalysisException: Input slot(s) not in child's output: k5#5 in plan: LogicalProject[176] ( distinct=false, projects=[(cast(k5#5 as DECIMALV3(16, 10)) / k5#5) AS `nu`#14, sum(k1)#15], excepts=[] ), child output is: [nu#16, sum(k1)#15]
plan tree:
LogicalProject[176] ( distinct=false, projects=[(cast(k5#5 as DECIMALV3(16, 10)) / k5#5) AS `nu`#14, sum(k1)#15], excepts=[] )
+--LogicalAggregate[168] ( groupByExpr=[nu#16], outputExpr=[nu#16, sum(k1#1) AS `sum(k1)`#15], hasRepeat=false )
   +--LogicalProject[156] ( distinct=false, projects=[k1#1, (cast(k5#5 as DECIMALV3(16, 10)) / k5#5) AS `nu`#16], excepts=[] )
      +--LogicalOlapScan ( qualified=default_cluster:regression_test_nereids_syntax_p0.test, indexName=test, selectedIndexId=503229, preAgg=OFF, Aggregate function sum(k1) contains key column k1. )
    at org.apache.doris.nereids.rules.analysis.CheckAfterRewrite.checkAllSlotReferenceFromChildren(CheckAfterRewrite.java:108) ~[classes/:?]
```
2023-10-26 02:31:20 -05:00
bc606859c7 [enhancement](regression-test) add test for test_avg (#25892) 2023-10-26 14:33:49 +08:00
d6c64d305f [chore](log) Add log to trace query execution #25739 2023-10-26 14:09:25 +08:00
6ce1087c23 [enhancement](stats) Limit analyze info count (#25576)
Each analyze job info and task info would not exceed 20000 after this PR. User could adjust this by FE conf param: analyze_record_limit
2023-10-26 13:46:28 +08:00
d896debbe7 [fix](stats) Escape string when insert stats (#25815)
Co-authored-by: AKIHA <cyborgz1999@example.com>
2023-10-26 13:39:41 +08:00
af3e6f604a [fix](nereids) prune partition bug in pattern ColA <> ColB #25769
in predicate rewrite phase, we eliminate some conjuncts which contains un-interested columns.
for example: T (a, b) partition by (a)
interest cols: a
uninsterest cols: b
for parition prune,
filter "a=1 and a>b" is equivalent to "a=1",
filter "a=1 or a>b" is equivalent to "TRUE"
2023-10-26 12:06:37 +08:00
e7a3cb079b [Enhance](regression)docker hive s3 file address is determined based on the configuration (#25905)
docker hive s3 file address is determined based on the configuration custom_settings.env
2023-10-26 11:58:33 +08:00
9aa9894b11 [fix](Nereids) create table default value error msg is diff with lagacy planner (#25897)
old error msg:
default value precision: 2023-10-25 14:45:30.292 can not be greater than type precision: DATETIME(1)

new error msg:
default value precision: CURRENT_TIMESTAMP(3) can not be greater than type precision: DATETIME(1)
2023-10-25 22:36:14 -05:00
855956e503 [opt](Nereids) let column dist info check msg same with legacy planner (#25925) 2023-10-25 22:35:24 -05:00
005a36322e [opt](index compaction) optimize checks before index compaction (#25486) 2023-10-25 22:21:46 -05:00
6e1a4dbda2 [Fix](predicate pushdown) Common expression not acting on any slot should not be pushed down (#25901) 2023-10-26 11:20:12 +08:00
d7d284b4f8 [fix](Nereids) fix '' and "" in string literal (#25752)
'' represents ' in sql standard
"" represents " in sql standard
2023-10-26 11:18:21 +08:00
f322ad277d [fix](nereids) PushdownAliasThroughJoin may lost required columns from parent node (#25835) 2023-10-25 22:03:41 -05:00
31d2a9a4f5 [Enhancement](function) support fractions for convert_tz(datetimev2) (#25915)
mysql> select convert_tz('2019-08-01 01:01:02.123' , '+00:00', '+07:00');
+----------------------------------------------------------------------------------+
| convert_tz(cast('2019-08-01 01:01:02.123' as DATETIMEV2(3)), '+00:00', '+07:00') |
+----------------------------------------------------------------------------------+
| 2019-08-01 08:01:02.123                                                          |
+----------------------------------------------------------------------------------+
1 row in set (0.18 sec)
2023-10-26 10:46:47 +08:00
ad1313cce6 [refactor](pipelineX) refine _build_side_pipelines (#25871) 2023-10-26 10:32:23 +08:00
78165a3f6b [refine](pipelineX) use finish dependency in task (#25881) 2023-10-26 10:32:13 +08:00
6dd60c6ebb [Enhance](BE) Add -Wshadow-field compile option to avoid unexpected shadowing behavior (#25698)
* Fix `Tablet::_meta_lock` shadows member inherited from `BaseTablet`

* Add -Wshadow-field compile option to avoid unexpected shadowing behavior
2023-10-26 10:00:28 +08:00
da4de17d5c [improvement](function) improve date_trunc function performance when timeunit is const (#25824)
this PR #22602 have check function.
only support date_trunc(column, const), so the second must be const literal
and no need to check time unit every row.
2023-10-26 09:51:21 +08:00
77f727e0a1 [chore](compaction) Print roswet size when compaction finishes successfully (#25891) 2023-10-26 09:49:28 +08:00
a079ba763c [fix](tvf) fix 'frontends()' inconsistent with return 'show proc /fro… (#25885) 2023-10-26 09:47:58 +08:00
4434b3f32e [fix](move-memtable) use pthread mutex in LoadStreamMgr (#25882) 2023-10-26 09:19:59 +08:00
f3e6bcebd7 [github](action) re-add "Add Scope Labeler" and add new label "meta-change" (#25907)
---------

Co-authored-by: stephen <hello-stephen@qq.com>
2023-10-26 09:17:22 +08:00
77bea97d40 [test](sync)add sync after insert in test case (#25911) 2023-10-26 09:11:25 +08:00