database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
11,172 Commits 1 Branch 0 Tags
4ac38ca67adce44ba2e8a574d052dcd6ea342a11
Commit Graph

23 Commits

Author SHA1 Message Date
Jerry Hu
e90b78d783 [chore](regression) add case in test_delete (#20372) 
Add some cases of deletion conditions with numeric values.
 2023-06-05 09:38:29 +08:00
Jerry Hu
8ff8705b3f [fix](olap) deletion statement with space conditions did not take effect (#20349) 
Deletion statement like this:

delete from tb where k1 = '  ';
The rows whose k1's value is ' ' will not be deleted.
 2023-06-02 13:52:57 +08:00
amory
ee34b6de2d [Refact] (serde) refact mysql serde with data type (#19543) 
refact mysql output (de)serialize with data type serde , avoid accoriding switch case Primitive type writed in mysqlWriter
 2023-05-26 14:11:17 +08:00
HappenLee
473f0c45ff [Bug](delete) Fix bug of delete partition prune error (#18057) 2023-03-24 20:22:12 +08:00
morrySnow
173d68409c [enhencement](planner) update and delete support use alias for target table (#17914) 2023-03-22 11:07:39 +08:00
morrySnow
0086fdbbdb [enhancement](planner) support delete from using syntax (#17787) 
support syntax delete using, this syntax only support UNIQUE KEY model

use the result of `t2` join `t3` to romve rows from `t1`

```sql
-- create t1, t2, t3 tables
CREATE TABLE t1
  (id INT, c1 BIGINT, c2 STRING, c3 DOUBLE, c4 DATE)
UNIQUE KEY (id)
DISTRIBUTED BY HASH (id)
PROPERTIES('replication_num'='1', "function_column.sequence_col" = "c4");

CREATE TABLE t2
  (id INT, c1 BIGINT, c2 STRING, c3 DOUBLE, c4 DATE)
DISTRIBUTED BY HASH (id)
PROPERTIES('replication_num'='1');

CREATE TABLE t3
  (id INT)
DISTRIBUTED BY HASH (id)
PROPERTIES('replication_num'='1');

-- insert data
INSERT INTO t1 VALUES
  (1, 1, '1', 1.0, '2000-01-01'),
  (2, 2, '2', 2.0, '2000-01-02'),
  (3, 3, '3', 3.0, '2000-01-03');

INSERT INTO t2 VALUES
  (1, 10, '10', 10.0, '2000-01-10'),
  (2, 20, '20', 20.0, '2000-01-20'),
  (3, 30, '30', 30.0, '2000-01-30'),
  (4, 4, '4', 4.0, '2000-01-04'),
  (5, 5, '5', 5.0, '2000-01-05');

INSERT INTO t3 VALUES
  (1),
  (4),
  (5);

-- remove rows from t1
DELETE FROM t1
  USING t2 INNER JOIN t3 ON t2.id = t3.id
  WHERE t1.id = t2.id;
```

the expect result is only remove the row where id = 1 in table t1

```
+----+----+----+--------+------------+
| id | c1 | c2 | c3     | c4         |
+----+----+----+--------+------------+
| 2  | 2  | 2  |    2.0 | 2000-01-02 |
| 3  | 3  | 3  |    3.0 | 2000-01-03 |
+----+----+----+--------+------------+
```
 2023-03-16 13:12:00 +08:00
amory
06dee69174 [Refactor](map) remove using column array in map to reduce offset column (#17330) 
1. remove column array in map 
2. add offsets column in map 
Aim to reduce duplicate offset  from key-array and value-array in disk
 2023-03-09 11:22:26 +08:00
xueweizhang
9155d8b9d1 [fix](delete) fix 'is null' or 'is not null' delete predicate will get wrong result (#17190) 
fix 'is null' or 'is not null' delete predicate will get wrong result

Signed-off-by: nextdreamblue <zxw520blue1@163.com>
 2023-03-02 14:05:44 +08:00
xueweizhang
e0cd8599d2 [fix](delete) fix delete from bug which can get wrong result (#17146) 
理论上，如果是两次独立的删除，比如delete from table where a=1; delete from table where a=2;其实这个地方应该可以使用的，但是目前的代码，是把所有不同版本的delete predicates和不同列的delete predicates都放到一起了，失去了版本信息、失去了谓词间可能是and的关系，统一弱化成了delete predicates都是独立的，有一个delete predicates满足条件，就把page都去掉。
这个pr的修改方式，就是在当前代码的基础上，当只有一个delete predicate的时候才能保证后续淘汰page的正确性，所以这里一律加了 == 1的判断才传递delete predicates。
如果要把不同版本的delete predicates和不同列的delete predicates作为完整和严谨的逻辑去判断page，需要修改的设计就有点多了，目前的方案算是一种优先解决bug的思路，后续可以进一步把delete predicates这块加速zone判断进行page淘汰的逻辑完善，提高delete predicates使用的场景。
 2023-02-28 09:20:10 +08:00
xy720
91fc9fae8e [Bug](complex-type) Fix is null predicate in delete stmt for array/struct/map type (#17018) 2023-02-23 15:06:49 +08:00
Gabriel
4dbe30d37b [regression](vectorized) delete vectorized config in regression tests (#15126) 2022-12-16 17:08:29 +08:00
Gabriel
5ef4c42a80 [Bug](datev2) Fix wrong result when use datev2 as partition key (#15094) 2022-12-15 21:27:05 +08:00
yuanyuan8983
442b844b22 [regressiontest](delete)delete-where-in-test (#14036) 
* delete-where-in-test

* Update test_delete_where_in.groovy

* Update test_delete_where_in.groovy
 2022-11-15 18:35:31 +08:00
Gabriel
42b2725f03 [Bug](delete) Fix wrong delete operation (#13840) 2022-11-01 13:38:43 +08:00
Ashin Gau
45b31506c7 [improvement](delete) support delete from partitioned table without partition specified (#13533) 
Support delete from partitioned table without partition specified in [DELETE] stmt.

## Usage
If it is a partitioned table, you can specify a partition.
If not specified, Doris will infer partition from the given conditions.
In two cases, Doris cannot infer the partition from conditions:
1) the conditions do not contain partition columns;
2) The operator of the partition column is `not in`.
When a partition table does not specify the partition,
or the partition cannot be inferred from the conditions,
the session variable `delete_without_partition` needs to be `true`
to make delete statement be applied to all partitions.

## Test case
Test case is added in `regression-test/suites/delete_p0/test_delete_from_partition.groovy`,
user can delete from partitioned table without partition specified now.
 2022-10-27 21:32:45 +08:00
camby
1f7829e099 [Fix](array-type) bugfix for array column with delete condition (#13361) 
Fix for SQL with array column:
delete from tbl where c_array is null;

more info please refer to #13360

Co-authored-by: cambyzju <zhuxiaoli01@baidu.com>
 2022-10-21 09:29:02 +08:00
Xin Liao
bbb6d2758a [fix](regression-test) fix test_segment_iterator_delete using order_qt_sql (#13192) 2022-10-09 11:35:22 +08:00
Yongqiang YANG
6b6d548df9 [enhancement](test) add more p0 cases (#12285) 2022-09-29 10:45:17 +08:00
Yongqiang YANG
fb27e3ef31 [fix](planner) let OlapScanNode turn off preaggragation when there is a filter on DELETE_SIGN (#12118) 
We can skip aggregate on replace column, otherwise it would generate
wrong result. e.g. a row in UNIQUE is deleted by delte_sign_column,
then it would be returned.
 2022-08-30 15:54:37 +08:00
Gabriel
1f9eec5462 [Regression](datev2) Add test cases for datev2/datetimev2 (#11831) 2022-08-19 10:57:55 +08:00
luozenglin
0f75bd0e38 [fix](delete) fix query result error after delete (#11754) 
convert dictionary code for delete predicates.
 2022-08-15 17:52:03 +08:00
yiguolei
ea57bf6370 [refactor](delete predicate) Unify delete to segmentiterator (#11650) 
* remove seek columns and unify delete columns in rowset reader


Co-authored-by: yiguolei <yiguolei@gmail.com>
 2022-08-11 15:12:43 +08:00
Yongqiang YANG
ee4d9d4347 [improvement](test) group some cases and group a case to p0 if it is not grouped (#11548) 2022-08-06 15:12:08 +08:00