4f5db35990
[fix](date) fix the value may be changed during the parsing of date and datetime types ( #11573 )
...
* [fix](date) fix the value may be changed during the parsing of date and datetime types
2022-08-08 08:58:30 +08:00
8802a41918
fix profile may cause query slow ( #11386 )
...
Co-authored-by: Wang Bo <wangbo36@meituan.com >
2022-08-07 20:52:52 +08:00
683a1261c6
[Enhancement](vectorized) Runtime Filter support equivalent slot of outer join ( #11530 )
2022-08-06 08:10:28 +08:00
57b7a416d2
[chore](build) add apache snapshot maven repo to repositories ( #11549 )
2022-08-06 07:15:28 +08:00
3070318f95
[Enhancement](IdGenerator) Use IdGeneratorBuffer to get better performance for creating tablet in fe when do alter table job ( #11524 )
...
Co-authored-by: caiconghui1 <caiconghui1@jd.com >
2022-08-05 23:27:29 +08:00
d88d1239c5
[feature] (Nereids) support limit clause ( #11209 )
...
including:
1. limit clause parser
2. implementation rule to transform LogicalLimit to PhysicalLimit
2022-08-05 11:58:45 +08:00
6eb8ac0ebf
[feature-wip][multi-catalog]Support caseSensitive field name in file scan node ( #11310 )
...
* Impl case sentive in file scan node
2022-08-05 08:03:16 +08:00
d4e6e3edfd
[bugfix]fix time accuracy ( #11521 )
2022-08-04 21:36:20 +08:00
e11024f5cc
[enhancement](Nereids)set default join type to CROSS_JOIN ( #11459 )
...
set default join type to CROSS_JOIN on join that has no equal on condition when parse sql string.
2022-08-04 21:25:24 +08:00
6dc41d57f3
[enhancement](Nereids)support count, min and avg function ( #11374 )
...
1. add count function
2. add min function
3. add avg function
2022-08-04 21:19:32 +08:00
591b7f3f92
[multi-catalog](oss)Support hive external table on Ali oss. ( #11489 )
2022-08-04 17:44:34 +08:00
95091256b0
[chore](deps) update bdbje tp doris bdbje, update libhdfs3 to improve performance ( #11497 )
2022-08-04 17:10:56 +08:00
9f221a703b
[feature-wip](statistics) step5: show statistics job information ( #8862 )
...
This pull request includes some implementations of the statistics(https://github.com/apache/incubator-doris/issues/6370 ), it will not affect any existing code and users will not be able to create statistics job.
It implements the display of statistics job information, with the following syntax, users will be able to view the corresponding job information.
syntax:
```
SHOW ANALYZE
[TABLE | ID]
[
WHERE
[STATE = ["PENDING"|"SCHEDULING"|"RUNNING"|"FINISHED"|"FAILED"|"CANCELLED"]]
]
[ORDER BY ...]
[LIMIT limit][OFFSET offset];
```
e.g.
| id | create_time | start_time | finish_time | error_msg | scope | progress | state |
| ----- | ----------------------- | ----------------------- | ----------------------- | --------- | ------------------- | -------- | -------- |
| 60051 | 2022-07-21 01:26:26.173 | 2022-07-21 01:26:26.186 | 2022-07-21 01:26:27.104 | | table1(citycode,pv) | 5/5 | FINISHED |
2022-08-04 16:10:49 +08:00
397bf354db
[WIP](optional) using hash set to distinct single value ( #11246 )
...
* [WIP](optional) using hash set to distinct single value
Co-authored-by: wangbo36@meituan.com <wangbo36@meituan.com >
2022-08-04 15:52:58 +08:00
9078ab4d24
[feature](FE): add new property to control whether use light schema change or not ( #11169 )
2022-08-04 15:49:05 +08:00
ec3c911f97
[Feature][Materialized-View] support materialized view on vectorized engine ( #10792 )
2022-08-04 14:07:48 +08:00
e7f378fec6
[Enhancement](IdGenerator) Use IdGeneratorBuffer to get better performance for getNextId operation when create table, truncate table, add partition and so on ( #11479 )
...
Co-authored-by: caiconghui1 <caiconghui1@jd.com >
2022-08-04 11:21:35 +08:00
7703912b3e
[improvement](error msg)improve the err msg returned when the key not in columns when create table ( #11490 )
2022-08-04 11:20:49 +08:00
ce68d24e95
[Bug](function) fix current_date not equal to curdate ( #11463 )
...
* fix current_date not equal to curdate
2022-08-04 09:25:50 +08:00
33053ad1fe
[improvement](outfile) support multibyte separator in outfile clause ( #11487 )
2022-08-04 09:06:06 +08:00
4ba2422039
[improvement](fe) Remove constant keys in aggregation ( #11434 )
2022-08-03 19:43:35 +08:00
a47eff1e46
[enhancement](Nereids) support all join type in Nereids that could do join by HashJoinNode ( #11446 )
...
add and test join type:
1. inner join
2. left outer join
3. right outer join
4. left semi join
5. right semi join
6. left anti join
7. right anti join
2022-08-03 12:14:17 +08:00
5b9b6c9065
[WIP](decimalv3) WIP ( #11443 )
...
* [feature-WIP](decimalv3) fix some bugs of decimalv3
2022-08-03 11:21:36 +08:00
77d82bb292
[Bug](MaterializedView) Fix bug of light schema change do not set right unique id cause MV coredump ( #11396 )
...
Fix bug of light schema change do not set right unique id cause MV coredump
2022-08-03 11:21:28 +08:00
4ae4909a2b
[refactor](tvf) table-valued-function table ( #11452 )
2022-08-03 10:39:16 +08:00
c581855a41
[fix](hive-table) fix bug that hive external table can not query table created by Tez ( #11345 )
...
* [fix](hive-table) fix bug that hive external table can not query table created by Tez
If the hive is created by Tez, the location of the table is a second-level director, eg:
/user/hive/warehouse/region_tmp_union_all/
---/user/hive/warehouse/region_tmp_union_all/1
---/user/hive/warehouse/region_tmp_union_all/2
We should recursive traverse the directory to get the real files.
2022-08-03 09:07:47 +08:00
db3ba02993
[fix](planner) Fix an issue where outputSmap's size could grow exponentially ( #11378 )
...
* [fix](planner) Fix an issue where outputSmap'size could grow exponentially
2022-08-03 09:07:00 +08:00
1e59c4054a
[minor](log) add some log to observe the change of table's state. ( #11448 )
...
When upgrading to 1.x, some table's state may change to ROLLUP,
Resulting in not able to create/drop/modify partition.
I haven't find the root cause yet, so I add some log to observe
the change of table's state.
2022-08-03 08:43:14 +08:00
d6149e4777
[Enhancement](TabletInvertIndex) use StampLock to instead of ReentranReadWriteLock to get better performance ( #11422 )
...
Co-authored-by: caiconghui1 <caiconghui1@jd.com >
2022-08-02 17:53:30 +08:00
ff8a4ec2f2
[WIP][feature] support create,alter,refresh,drop stmt syntax for multi table materialized view ( #11218 )
...
* [WIP][feature] support `create`,`alter`,`refresh`,`drop` stmt for multi
table materialized view
2022-08-02 16:03:36 +08:00
cd6fbd09bf
[feature] (Nereids) add rule to merge consecutive project nodes ( #11262 )
...
Merge consecutive project nodes. For example:
logical plan tree:
```
project(a)
|
project(a,b)
|
project(a, b, c)
|
scan
```
transformed to:
```
project(a)
|
scan
```
2022-08-02 15:58:55 +08:00
e41bfd8149
[Bug] Update Stmt Fail use in predicate query ( #11414 )
...
Co-authored-by: lihaopeng <lihaopeng@baidu.com >
2022-08-02 15:14:36 +08:00
fcfc76f4a9
[Bug](date function) from_unixtime return wrong result ( #11410 )
...
* [Bug](date function) `from_unixtime` return wrong result
2022-08-02 14:23:54 +08:00
f730a048b1
[feature-wip](load) Support single replica load ( #10298 )
...
During load process, the same operation are performed on all replicas such as sort and aggregation,
which are resource-intensive.
Concurrent data load would consume much CPU and memory resources.
It's better to perform write process (writing data into MemTable and then data flush) on single replica
and synchronize data files to other replicas before transaction finished.
2022-08-02 11:44:18 +08:00
abbf75d302
[doc][refactor](metrics) Reorganize FE and BE metrics and add document ( #11307 )
2022-08-02 11:34:06 +08:00
8e3979b3d1
FIX: audit user ( #11408 )
2022-08-02 11:19:02 +08:00
6af35a8eb7
[fix](TabletInvertedIndex) fix potential deadlock between ForkJoinPool and TabletInvertedIndex ( #11365 )
...
* [fix](TabletInvertedIndex) fix potential deadlock between ForkJoinPool and TabletInvertedIndex
The default ForkJoinPool is shared by all parallelStream by default, and we obtain read lock outside the ForkJoinPool in TabletInvertIndex while we obtain read lock inside the same ForkJoinPool in TabletStatMgr which may cause deadlock
2022-08-02 10:08:05 +08:00
27be5e8667
[feature-wip](decimalv3) Fix UTs when decimalv3 is enabled ( #11380 )
2022-08-01 23:07:38 +08:00
dddacd120d
[fix](planner) Fix inconsistent nullability between outputTuple and groupByExpr when executing agg query ( #11361 )
2022-08-01 22:57:51 +08:00
80ce027ea2
[refactor](Nereids): refactor UT of Nereids ( #11330 )
...
refactor the UT of Nereids.
Extract the plan constructor (This PR extract all olapscan and table into PlanConstructor).
2022-08-01 22:53:00 +08:00
69961ac19c
fix bug for lower case when rename table name ( #11403 )
2022-08-01 22:30:41 +08:00
6da6ffb836
[feature] (Nereids) add rewrite rule to merge consecutive filter nodes ( #11248 )
...
this rule aims to merge consecutive filters.
For example:
logical plan tree:
```
project
|
filter(a>0)
|
filter(b>0)
|
scan
```
transformed to:
```
project
|
filter(a>0 and b>0)
|
scan
```
2022-08-01 17:29:15 +08:00
69bfbae856
[enhancement](nereids) Normalize expressions before performing plan rewriting ( #11299 )
...
Rules for normalizing expressions should be applied once before do some extra expression transforms.
Normalization rules include:
1. NormalizeBinaryPredicatesRule
2. BetweenToCompoundRule
3. SimplifyNotExprRule
2022-08-01 17:15:04 +08:00
7ca6f8b6a2
[refactor](Nereids)split rewrite and insert into memo to 2 functions ( #11342 )
...
Split rewrite and insert into memo to 2 functions to make the code easy to read.
2022-08-01 14:44:01 +08:00
1cf57a985d
[fix] Fix the query result error caused by the grouping sets statemen… ( #11316 )
...
* [fix] Fix the query result error caused by the grouping sets statement grouping as an expression
2022-08-01 13:52:18 +08:00
0325fa436e
[fix](agg)Add field of 'is_first_phase' in TAggregationNode ( #11321 )
2022-08-01 11:49:50 +08:00
34ed40bb71
[Feature](array) doe support array ( #11340 )
2022-08-01 10:56:40 +08:00
76970200ec
[Bugfix](light-schema-change) Fix bf columns message miss when load ( #11367 )
2022-08-01 10:53:25 +08:00
547c18d28b
[feature](load) support CLEAN LABEL stmt ( #11362 )
2022-08-01 10:43:33 +08:00
3f43d7262b
[Bug] (datetimev2) fix bugs for datev2/datetimev2 ( #11358 )
2022-08-01 10:29:56 +08:00
