Commit Graph

530 Commits

Author SHA1 Message Date
e6080a6e4c [regression](join) add right anti join with other predicate regression case (#13815) 2022-11-02 13:27:58 +08:00
277025b046 [fix](join)ColumnNullable need handle const column with nullable const value (#13866) 2022-11-02 08:52:49 +08:00
287a739510 [javaudf](string) Fix string format in java udf (#13854) 2022-11-01 21:25:12 +08:00
61c817f4cc [feature](syntax) support SELECT * EXCEPT (#13844)
* [feature](syntax) support SELECT * EXCEPT: add regression test
2022-11-01 19:41:25 +08:00
e63608b556 [Bug](test) fix some test case result is ramdom (#13837) 2022-11-01 14:06:47 +08:00
42b2725f03 [Bug](delete) Fix wrong delete operation (#13840) 2022-11-01 13:38:43 +08:00
b27714542d [fix](planner) infer predicate could generate predicates in another scope (#13691)
* [fix](planner) infer predicate could generate predicates in another scope
2022-11-01 09:03:41 +08:00
d2c5c1af3b [feature](regression) add custom config file for Regression: regression-conf-custom.groovy (#13783) 2022-10-31 22:49:06 +08:00
ba177a15cb [feature-wip](recover) new recover ddl and support show catalog recycle bin (#13067) 2022-10-31 17:44:56 +08:00
f49a0daf54 [fix](regression) Fix concurrent regression failure (#13798) 2022-10-31 15:57:45 +08:00
53e5f3939e [fix](plan)result exprs should be substituted in the same way as agg exprs (#13744)
* [fix](cast)ignore implicit cast when comparing two exprs

* fix fe ut
2022-10-31 10:19:32 +08:00
61b7c2c96c [fix](join) fix incorrect result when using anti join with other join predicates (#13743) 2022-10-31 09:51:34 +08:00
b15e0a9fb5 [Bug](function) fix bug of if function of nullable column process (#13779) 2022-10-31 08:38:53 +08:00
753c2ccfd1 [fix](test) drop table before create it (#13791) 2022-10-30 22:45:08 +08:00
98cc32aa0e [BugFix](regression-test) add order by in left/right join test case (#13774)
The result of right join is unordered, so we need add order by to guarantee results consistent.
2022-10-30 18:00:08 +08:00
efe813ba60 [fix](test)(explain) add full qualified name for scan node explain string (#13777)
1.
In the "explain" result of SQL, the table name in `ScanNode` should be full qualified with dbname.
And for olap scan node, the selected index name should not be "null".

2.
Remove `tpch_sf1_p1/tpch_sf1/nereids/` in regression test, it will be fixed later.
2022-10-30 13:24:48 +08:00
f325119362 [fix](regression-test) update table name string in tpch_sf1 explain case (#13724) 2022-10-28 11:13:29 +08:00
5805011629 [Feature](string-function) Add function mask/mask_first_n/mask_last_n (#13694)
Implementation of mask function from hive.
2022-10-28 10:43:56 +08:00
20363edc73 [BugFix](function) fix reverse function dynamic buffer overflow due to illegal character (#13671)
Previous logic of reverse function might not be strong enough to handle illegal character. For example, one one byte size character would be mistaken as one utf-8 character which occupies more than one byte space. And unfortunately exceeding the buffer space during future process.
2022-10-28 08:44:08 +08:00
859ffa6304 [bugfix](concat) be crash caused by function concat(ifnull) (#13693) 2022-10-28 08:42:51 +08:00
c108554f14 [function](date function) add new date function 'to_monday' #13707 2022-10-28 08:41:16 +08:00
43c6428aea [Function](string) support sub_replace function (#13736)
* [Function](string) support sub_replace function

* remove conf
2022-10-28 08:40:08 +08:00
36053d2419 [fix](array-type) fix the be core dump when select the invalid array format (#13514)
1. this pr is used to fix the be core dump when select the invalid array.
2. before the change, we run "select array_intersect([1, 2, 3, 1, 2, 3], '1[3, 2, 5]');" will cause be core dump.
MySQL [example_db]> select array_intersect([1, 2, 3, 1, 2, 3], '1[3, 2, 5]');
ERROR 1105 (HY000): RpcException, msg: io.grpc.StatusRuntimeException: UNAVAILABLE: Network closed for unknown reason
3. after the change, we run "select array_intersect([1, 2, 3, 1, 2, 3], '1[3, 2, 5]');" will get error message.
MySQL [example_db]> select array_intersect([1, 2, 3, 1, 2, 3], '1[3, 2, 5]');
errCode = 2, detailMessage = No matching function with signature: array_intersect(array<tinyint(4)>, varchar(-1))"
Co-authored-by: hucheng01 <hucheng01@baidu.com>
2022-10-27 23:11:12 +08:00
45b31506c7 [improvement](delete) support delete from partitioned table without partition specified (#13533)
Support delete from partitioned table without partition specified in [DELETE] stmt.

## Usage
If it is a partitioned table, you can specify a partition.
If not specified, Doris will infer partition from the given conditions.
In two cases, Doris cannot infer the partition from conditions:
1) the conditions do not contain partition columns;
2) The operator of the partition column is `not in`.
When a partition table does not specify the partition,
or the partition cannot be inferred from the conditions,
the session variable `delete_without_partition` needs to be `true`
to make delete statement be applied to all partitions.

## Test case
Test case is added in `regression-test/suites/delete_p0/test_delete_from_partition.groovy`,
user can delete from partitioned table without partition specified now.
2022-10-27 21:32:45 +08:00
738da0b139 [bugfix](join) inner join return wrong result (#13608)
* bug fix for vhash join

* add regression test

Co-authored-by: cambyzju <zhuxiaoli01@baidu.com>
2022-10-27 11:48:41 +08:00
c874931ac8 [fix](join)output all value from no-null side of outer join (#13655)
* [fix](joinoutput all value from no-null side of outer join

* add regression test
2022-10-27 10:48:36 +08:00
7557980d64 [improvement](regression-test) avoid query empty result after loading finished (#13682)
When running regression test, we always found that the query return empty result after loading finished,
even if we call "sync" before the query.
This is because for `stream load`, the load task result will be returned immediately after the txn's status changed to VISIBLE,
but before writing the edit log.
So if we do the query right after we got the load task result, it is possible that we can not see the latest loaded data.

Same issue with `insert` operation
2022-10-27 09:47:18 +08:00
5bd66243ee [minor](log) remove some unused logs (#13689)
1. When running regression test with specific suites or group, do not print other suite name or file name
2. Remove unused alter table job log.
2022-10-27 09:37:32 +08:00
bed759b3f5 [Fix](array-type) support CTAS for ARRAY column from collect_list and collect_set (#13627)
Co-authored-by: cambyzju <zhuxiaoli01@baidu.com>
2022-10-26 19:42:15 +08:00
3548d0b824 [fix](statistics) fix cross join statistics exception (#13645) 2022-10-26 14:10:57 +08:00
c418bbd2d1 [feature-wip](new-scan) support Json reader (#13546)
Issue Number: close #12574
This pr adds `NewJsonReader` which implements GenericReader interface to support read json format file.

TODO:
1. modify `_scann_eof` later.
2. Rename `NewJsonReader` to `JsonReader` when `JsonReader` is deleted.
2022-10-26 12:52:21 +08:00
15130c469f [fix](planner) cannot recogonize column's table when analyze rewrite expr (#13597)
We save mv column with alias as table name, and search it with original table name.
2022-10-26 11:15:48 +08:00
e5b33abd3c [fix](planner) inlineView alias error (#13600) 2022-10-26 10:14:04 +08:00
295d887cf5 [improvement](thread) set name for priority thread pool (#13552) 2022-10-26 09:32:15 +08:00
e00734348b [Chore](regression) Fix wrong result for decimal (#13644) 2022-10-26 09:24:46 +08:00
6f18726f01 [improvement](test) add sync for test_agg_keys_schema_change_datev2 (#13643)
1. add "sync" to avoid some potential meta sync problem when running regression test on multi-node cluster
2. Use /tmp dir as dest dir of outfile test, to avoid "No such file or directory" error.
2022-10-25 22:29:05 +08:00
f209b7ab6e [fix](Nereids) add exchange node check between local and global agg in plan translator (#12913)
### table schema
CREATE TABLE `t1` (
  `k1` int(11) NULL,
  `v1` int(11) NULL
) ENGINE=OLAP
DUPLICATE KEY(`k1`, `v1`)
COMMENT 'OLAP'
DISTRIBUTED BY HASH(`k1`)
BUCKETS 3
PROPERTIES('replication_num'='1')

### query
select k1,count(distinct v1+1) from t1 group by k1;

### error
java.lang.ClassCastException: org.apache.doris.planner.OlapScanNode cannot be cast to org.apache.doris.planner.AggregationNode
2022-10-25 16:55:29 +08:00
e103531e69 [fix](sort)order by constant expr bug (#13613)
Issue Number: close (#13350)
2022-10-25 16:43:18 +08:00
b85c78ee00 [fix](regression) add 'if not exists' to 'create table' to support parallel test (#13576) (#13578)
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
2022-10-25 16:37:07 +08:00
235c105554 [feature-array](array-type) Add array function array_enumerate (#13612)
Add array function array_enumerate
2022-10-25 15:12:11 +08:00
de5bc6a8a5 [fix](regression-test) set label for stream load (#13620) 2022-10-25 14:13:24 +08:00
f802fc37ff add date function 'last_day' (#13609) 2022-10-25 13:46:16 +08:00
57f479d9d2 [fix](test) let cases use their own table name (#13602) 2022-10-25 13:45:01 +08:00
7fe7c01125 [Bug](decimal) Fix incorrect result for decimal multiply (#13591)
Fix incorrect result for decimal multiply
2022-10-25 12:08:49 +08:00
7faad9f004 [FIX](agg)fix group by constant child expr bug (#13485) 2022-10-24 16:32:36 +08:00
40e122e5ef [fix](join)the build and probe expr should be calculated before converting input block to nullable (#13436)
* [fix](join)the build and probe expr should be calculated before converting input block to nullable

* remove_nullable can be called on const column
2022-10-24 14:50:06 +08:00
177e82bdab [Enhancement](array-type) Add type derivation for array functions (#13534)
From now, we don't support type derivation for array function's arguments.
So that the cases below will return wrong values or even cause be core.

mysql> select array_union([1],[10000000]);
+----------------------------------------+
| array_union(ARRAY(1), ARRAY(10000000)) |
+----------------------------------------+
| [1, -128]                              |
+----------------------------------------+
1 row in set (0.03 sec)

mysql> select array_union([NULL],[1]);
ERROR 1105 (HY000): RpcException, msg: io.grpc.StatusRuntimeException: UNAVAILABLE: Network closed for unknown reason

mysql> select array_union([],[1]);
ERROR 1105 (HY000): RpcException, msg: io.grpc.StatusRuntimeException: UNAVAILABLE: Network closed for unknown reason
This commit make a small fix to derivate the argument types of the array function
1、 For null type in arguments, cast the null type to boolean type, because null type should not be seen in be.
2、For different types in arguments, cast all arguments type to their compatible type.
2022-10-24 11:51:47 +08:00
e17c2416f0 [fix](join) fix be core dump when using right join with other join predicates (#13511) 2022-10-24 10:35:07 +08:00
4b5a2c1a65 [fix](export)(outfile) fix bug that export may fail when writing SUCCESS file (#13574) 2022-10-23 13:02:49 +08:00
a7c221d04e [Bug](sort) Fix bug in string sorter (#13548) 2022-10-22 21:26:23 +08:00