72a05a4358
[Bug](date) remove MinuteFloor/MinuteCeil for datev2 ( #16247 )
2023-02-01 14:57:51 +08:00
1b99746355
[Bug](function) enchance esquery error msg && forbid to_quantile_state #16274
...
forbidden to_quantile_state temporary to avoid core dump. waiting for [Feature] support QuantileState in vectorized engine #15868 get the ball rolling on implementation.
2023-02-01 14:06:09 +08:00
17bec356a3
[Bug](decimalv3) always use decimalv3 for show create table ( #16295 )
2023-02-01 09:54:42 +08:00
cd457312e4
[Enhancement](grouping) Add a switch for users to force using alias name in group by and having clause ( #15748 )
2023-01-31 23:46:31 +08:00
c63a960df6
[fix](planner) create view generate wrong sql when sql contains multi count distinct ( #16092 )
...
If sql in create view has more than one count distinct, and write column name explicitly.
We will generate sql contains function multi_count_distinct.
It cannot be analyzed and all query containing this view will fail.
2023-01-31 23:42:53 +08:00
f798f60afd
[Fix](Nereids) fix incorrectly push down cast expression in hash join conjunctions ( #16216 )
2023-01-31 20:02:31 +08:00
644efb6437
[enhencement](lock) print table lock owner when failed to try lock ( #16186 )
2023-01-31 18:21:18 +08:00
e7cd85f147
[feature](Nereids): generate phyiscal plan in DPhyp ( #15264 )
2023-01-31 14:16:25 +08:00
00a598a839
[feature](cooldown) Decouple storage policy and resource ( #15873 )
2023-01-31 14:13:47 +08:00
a8a29427f6
[fix](multi catalog)Collect decimal and date type min max statistic value ( #16262 )
...
The min and max value of decimal and date columns in hive external table are incorrect,
this pr is to parse the min max value in HMS correctly.
2023-01-31 11:58:56 +08:00
471db80f69
[Bug](date) Fix invalid date ( #16205 )
...
Issue Number: close #15777
2023-01-31 10:08:44 +08:00
e7c1d81419
[fix](planner) Pushdown constant predicate to all scan node in the lieteral view. #16217
...
Before this PR, planner might push a constant FALSE predicate to the wrong scan nodes in the literal view, and make this predicate useless
2023-01-30 22:18:43 +08:00
c6014975a4
[fix](Nereids): fix AesDecrypt, AesEncrypt. ( #16213 )
2023-01-30 18:52:56 +08:00
c59a8cb15d
[refactor](remove unused code) remove log error hub ( #16183 )
...
---------
Co-authored-by: yiguolei <yiguolei@gmail.com >
2023-01-30 16:53:56 +08:00
fdc042bb39
[fix](vresultsink) BufferControlBlock may block all fragment handle threads ( #16231 )
...
BufferControlBlock may block all fragment handle threads leads to be out of work
modify include:
BufferControlBlock cancel after max timeout
StmtExcutor notify be to cancel the fragment when unexcepted occur
more details see issue #16203
2023-01-30 16:53:21 +08:00
342f3168b5
[fix](Nereids) return null when encryption is invalid rather than throw exception ( #16234 )
2023-01-30 16:52:42 +08:00
a9671b6dfd
[feature](agg)support two level-hash map in aggregation node ( #15967 )
2023-01-30 16:43:33 +08:00
6bebf92254
[fix][FE] fix be coredump when children of FunctionCallExpr is folded ( #16064 )
...
Co-authored-by: shizhiqiang03 <shizhiqiang03@meituan.com >
fix be coredump when children of FunctionCallExpr is folded
2023-01-30 15:25:00 +08:00
6254258357
[Feature](Nereids) eliminate project when child is empty relation ( #16210 )
2023-01-30 11:28:07 +08:00
5c00caa259
[refactor](Nereids) refactor BindSlotReference for easy merge all bind process in one rule ( #16156 )
2023-01-30 10:57:39 +08:00
bd1b7e190c
[fix](Nereids): fix field(). ( #16214 )
2023-01-30 10:55:02 +08:00
ec4a56922f
[enhancement](memory) reduce memory usage for failed broker loads ( #15895 )
...
* [enhancement](memory) reduce memory usage for failed broker loads
2023-01-30 10:22:31 +08:00
7d437d5706
[Bug](function) running_difference function coredump in regression test ( #16215 )
2023-01-30 09:58:27 +08:00
b8a7297109
[Enhancement](profile) fill user field for profile. ( #16212 )
...
Co-authored-by: wangxiangyu@360shuke.com <wangxiangyu@360shuke.com >
2023-01-30 09:15:02 +08:00
d56043ab5a
[feature-wip](MTMV) Support setting variables in query statement ( #16060 )
...
## Use case
```shell
mysql> CREATE TABLE t_user (
-> event_day DATE,
-> id bigint,
-> username varchar(20)
-> )
-> DISTRIBUTED BY HASH(id) BUCKETS 10
-> PROPERTIES ('replication_num' = '1');
Query OK, 0 rows affected (0.07 sec)
mysql> CREATE TABLE t_user_pv(
-> event_day DATE,
-> id bigint,
-> pv bigint
-> )
-> DISTRIBUTED BY HASH(id) BUCKETS 10
-> PROPERTIES ('replication_num' = '1');
Query OK, 0 rows affected (0.09 sec)
mysql> CREATE MATERIALIZED VIEW mv
-> BUILD IMMEDIATE REFRESH COMPLETE
-> KEY (username)
-> DISTRIBUTED BY HASH(username) BUCKETS 10
-> PROPERTIES ('replication_num' = '1')
-> AS SELECT /*+ SET_VAR(exec_mem_limit=1048576, query_timeout=3600) */ t1.username ,t2.pv FROM t_user t1 LEFT JOIN t_user_pv t2 on t1.id = t2.id;
Query OK, 0 rows affected (0.10 sec)
```
2023-01-30 01:05:41 +08:00
98649ec9f8
[fix](Nereids): Fix some functions error ( #16197 )
...
* fix bugs in regexp_extract_all
* fix rpad
* fix weekofday
* fix cryptor
* fix timestamp
* fix st_ function
2023-01-30 00:41:31 +08:00
7d648a94d0
[fix](Nereids): fix scalar_function A-F. ( #16209 )
...
* [fix](Nereids): fix scalar_function A-F.
* [Fix](regression-test)fix regression test framework cannot compare double value nan and inf.
* revert dround()
2023-01-30 00:37:34 +08:00
217db3e4c8
[refactor](built-in function) remove symbols for vectorized function ( #16189 )
...
* [refactor](built-in function) remove symbols for vectorized function
* update
* update
2023-01-29 21:30:09 +08:00
1db7882bb5
[Fix](Nereids): fix error of X-Z function for nereids ( #16171 )
2023-01-29 20:42:30 +08:00
1ec88cbff6
[fix](nereids) AggregationNode process null as key column in wrong way ( #16125 )
...
in AggregationNode, _merge_with_serialized_key_helper method should convert the key column to full column if the key column is null literal.
2023-01-29 20:12:07 +08:00
1ad6ef939b
[refactor](Nereids) use immutable collections as far as possible ( #16193 )
2023-01-29 16:48:58 +08:00
04ed83cb36
[fix](Nereids): remove DataV2Type in ConvertTz SIGNATURES ( #16170 )
...
* [fix](Nereids): remove `DataV2Type` in ConvertTz SIGNATURES
* remove it in doris_builtins_functions.py
2023-01-29 16:11:17 +08:00
abc50c6fe5
[enhance](Nereids): remove duplicated alias Function. ( #16187 )
2023-01-29 14:56:20 +08:00
c6bc0a03a4
[feature](Load)Suppot MySQL Load Data ( #15511 )
...
Main subtask of [DSIP-28](https://cwiki.apache.org/confluence/display/DORIS/DSIP-028%3A+Suppot+MySQL+Load+Data )
## Problem summary
Support mysql load syntax as below:
```sql
LOAD DATA
[LOCAL]
INFILE 'file_name'
INTO TABLE tbl_name
[PARTITION (partition_name [, partition_name] ...)]
[COLUMNS TERMINATED BY 'string']
[LINES TERMINATED BY 'string']
[IGNORE number {LINES | ROWS}]
[(col_name_or_user_var [, col_name_or_user_var] ...)]
[SET (col_name={expr | DEFAULT} [, col_name={expr | DEFAULT}] ...)]
[PROPERTIES (key1 = value1 [, key2=value2]) ]
```
For example,
```sql
LOAD DATA
LOCAL
INFILE 'local_test.file'
INTO TABLE db1.table1
PARTITION (partition_a, partition_b, partition_c, partition_d)
COLUMNS TERMINATED BY '\t'
(k1, k2, v2, v10, v11)
set (c1=k1,c2=k2,c3=v10,c4=v11)
PROPERTIES ("auth" = "root:", "strict_mode"="true")
```
Note that in this pr the property named `auth` must be set since stream load need auth. I will optimize it later.
2023-01-29 14:44:59 +08:00
eb7da1c0ee
[fix](datatype) fix some bugs about data type array datetimev2 and decimalv3 ( #16132 )
2023-01-29 14:26:08 +08:00
578a855b3e
[Bug](topn-opt) filter condition for analytic info for two phase read opt ( #16173 )
...
two phase read optimization should not be enabled when query has analytic info
2023-01-29 12:06:18 +08:00
ce487e2b11
[fix](Nereids): fix dceil() dfloor() ( #16174 )
2023-01-29 11:59:23 +08:00
35398ad8d9
[fix](multi-catalog)Use -1 for column_statistics internal table idx_id default value instead of null, for external catalog ( #16177 )
...
The internal statistic table column_statistics has a non-null field idx_id, the insert sql for hive table set the default value to NULL, which will failed to insert the result. Change it to -1.
2023-01-29 11:29:25 +08:00
2b5f95f08a
[Bug](function) remove datev2 signature of hour_ceil/hour_floor #16168
2023-01-29 11:27:56 +08:00
3151d94e9e
[fix](Nereids): fix Ceiling. ( #16164 )
2023-01-28 20:26:20 +08:00
da28d2faee
[deps](http)Upgrade springboot version to 2.7.8 ( #16158 )
...
* Upgrade springboot version to 2.7.8
* fix
2023-01-28 20:13:50 +08:00
c506b4a1e3
[bug](cooldown)add config for Cooldown Job
2023-01-28 19:58:50 +08:00
26fc7c8196
[Bug](decimalv3) fix BE crash for function if ( #16152 )
2023-01-28 19:37:50 +08:00
7e7fd5d049
[cleanup](fe) cleanup useless code. ( #16129 )
...
* [cleanup](Nereids): cleanup useless code.
* revert ErrorCode.java
2023-01-28 18:44:43 +08:00
49395390be
[bugfix](metareader) meta reader could not load image ( #16148 )
...
This bug is introduced by PR #16009 .
Co-authored-by: yiguolei <yiguolei@gmail.com >
2023-01-28 14:22:18 +08:00
b919cbe487
[ehancement](nereids) Enhancement for limit clause ( #16114 )
...
support limit offset without order by.
the legacy planner supoort this feature in PR #15218
2023-01-28 11:04:03 +08:00
1589d453a3
[fix](multi catalog)Support parquet and orc upper case column name ( #16111 )
...
External hms catalog table column names in doris are all in lower case,
while iceberg table or spark-sql created hive table may contain upper case column name,
which will cause empty query result. This pr is to fix this bug.
1. For parquet file, transfer all column names to lower case while parse parquet metadata.
2. For orc file, store the origin column names and lower case column names in two vectors, use the suitable names in different cases.
3. FE side, change the column name back to the origin column name in iceberg while doing convertToIcebergExpr.
2023-01-27 23:52:11 +08:00
a3cd0ddbdc
[refactor](remove broker scan node) it is not useful any more ( #16128 )
...
remove broker scannode
remove broker table
remove broker scanner
remove json scanner
remove orc scanner
remove hive external table
remove hudi external table
remove broker external table, user could use broker table value function instead
Co-authored-by: yiguolei <yiguolei@gmail.com >
2023-01-23 19:37:38 +08:00
ab04a458aa
[Enhancement](export) cancel all running coordinators when execute cancel-export statement. ( #15801 )
2023-01-22 23:11:32 +08:00
253445ca46
[vectorzied](jdbc) fix jdbc executor for get result by batch and memo… ( #15843 )
...
result set should be get by batch size2.
fix memory leak3.
2023-01-21 08:22:22 +08:00
