Commit Graph

28 Commits

Author SHA1 Message Date
7ba4cd764a [enhancement](array-function) array_position,array_contains,countequal which in FunctionArrayIndex handle target NULL (#14564)
in the previous, the result is:
```
mysql> select array_position([1, null], null);
+--------------------------------------+
| array_position(ARRAY(1, NULL), NULL) |
+--------------------------------------+
|                                 NULL |
+--------------------------------------+
1 row in set (0.02 sec)
```

but after this commit, the result become:
```
mysql> select array_position([1, null], null);
+--------------------------------------+
| array_position(ARRAY(1, NULL), NULL) |
+--------------------------------------+
|                                    2 |
+--------------------------------------+
1 row in set (0.02 sec)
```
2022-11-25 14:19:50 +08:00
bc699511d0 [Fix](array-function) fix array_distinct null values (#14544)
in the previous the result is:
```
mysql> select array_distinct([1,1,3,3,null, null, null]);
+-----------------------------------------------------+
| array_distinct(ARRAY(1, 1, 3, 3, NULL, NULL, NULL)) |
+-----------------------------------------------------+
| [1, 3, NULL, NULL, NULL]                            |
+-----------------------------------------------------+
1 row in set (0.00 sec)
```

after this fix, the result becomes:
```
mysql> select array_distinct([1,1,3,3,null, null, null]);
+-----------------------------------------------------+
| array_distinct(ARRAY(1, 1, 3, 3, NULL, NULL, NULL)) |
+-----------------------------------------------------+
| [1, 3, NULL]                                        |
+-----------------------------------------------------+
1 row in set (0.00 sec)
```
2022-11-24 19:07:28 +08:00
16d8a1853a [Bug](array-function) array set function not handle all null value (#14318) 2022-11-22 09:07:43 +08:00
b4aef889f2 [feature-array](array-function) add array constructor function array() (#14250)
* [feature-array](array-function) add array constructor function `array()`

```
mysql>  select array(qid, creationDate) from nested_c_2  limit 10;
+------------------------------+
| array(`qid`, `creationDate`) |
+------------------------------+
| [1000038, 20090616074056]    |
| [1000069, 20090616075005]    |
| [1000130, 20090616080918]    |
| [1000145, 20090616081545]    |
+------------------------------+
10 rows in set (0.01 sec)
```
2022-11-19 10:49:50 +08:00
f86886f8f5 [Feature](function) Support array_compact function (#14141) 2022-11-15 14:24:37 +08:00
93e5d8e660 [Vectorized](function) support bitmap_from_array function (#14259) 2022-11-15 01:55:51 +08:00
43490a33a5 [feature-array](array-type) Add array function array_with_constant (#14115)
Return array of constants with length num.

```
mysql> select array_with_constant(4, 1223);
+------------------------------+
| array_with_constant(4, 1223) |
+------------------------------+
| [1223, 1223, 1223, 1223]     |
+------------------------------+
1 row in set (0.01 sec)
```
co-authored-by @eldenmoon
2022-11-11 22:08:43 +08:00
2e29b15c6a [test](array function)add array_range function test (#14123)
* add array_range function test

* add array_range function test
2022-11-11 18:04:33 +08:00
7ffe88b579 [feature-array](array-type) Add array function array_popback (#13641)
Remove the last element from array.

```
mysql> select array_popback(['test', NULL, 'value']);
+-----------------------------------------------------+
| array_popback(ARRAY('test', NULL, 'value')) |
+-----------------------------------------------------+
| [test, NULL]                                        |
+-----------------------------------------------------+
```
2022-11-07 10:48:16 +08:00
d183199319 [Bug](array-type) Fix array product calculate decimal type return wrong result (#13794) 2022-11-03 17:26:34 +08:00
738da0b139 [bugfix](join) inner join return wrong result (#13608)
* bug fix for vhash join

* add regression test

Co-authored-by: cambyzju <zhuxiaoli01@baidu.com>
2022-10-27 11:48:41 +08:00
235c105554 [feature-array](array-type) Add array function array_enumerate (#13612)
Add array function array_enumerate
2022-10-25 15:12:11 +08:00
177e82bdab [Enhancement](array-type) Add type derivation for array functions (#13534)
From now, we don't support type derivation for array function's arguments.
So that the cases below will return wrong values or even cause be core.

mysql> select array_union([1],[10000000]);
+----------------------------------------+
| array_union(ARRAY(1), ARRAY(10000000)) |
+----------------------------------------+
| [1, -128]                              |
+----------------------------------------+
1 row in set (0.03 sec)

mysql> select array_union([NULL],[1]);
ERROR 1105 (HY000): RpcException, msg: io.grpc.StatusRuntimeException: UNAVAILABLE: Network closed for unknown reason

mysql> select array_union([],[1]);
ERROR 1105 (HY000): RpcException, msg: io.grpc.StatusRuntimeException: UNAVAILABLE: Network closed for unknown reason
This commit make a small fix to derivate the argument types of the array function
1、 For null type in arguments, cast the null type to boolean type, because null type should not be seen in be.
2、For different types in arguments, cast all arguments type to their compatible type.
2022-10-24 11:51:47 +08:00
a555f45834 [fix](array-type) fix the wrong result of array_join function (#13477)
this pr is used to fix the wrong result of array_join function.
before the change, the array_join function will return wrong result.
MySQL [example_db]> select array_join(["", "1", "2"], '');
+--------------------------------------+
| array_join(ARRAY('', '1', '2'), '') |
+--------------------------------------+
| 1_2 |
+--------------------------------------+
3.after the change, the array_join function will return correct result.
MySQL [example_db]> select array_join(["", "1", "2"], '');
+--------------------------------------+
| array_join(ARRAY('', '1', '2'), '') |
+--------------------------------------+
| _1_2 |
+--------------------------------------+
Issue Number: #7570
2022-10-21 17:36:44 +08:00
3ca8bfaf30 [Function](array) support array_difference function (#13440) 2022-10-21 10:57:37 +08:00
32b1456b28 [feature-wip](array) remove array config and check array nested depth (#13428)
1. remove FE config `enable_array_type`
2. limit the nested depth of array in FE side.
3. Fix bug that when loading array from parquet, the decimal type is treated as bigint
4. Fix loading array from csv(vec-engine), handle null and "null"
5. Change the csv array loading behavior, if the array string format is invalid in csv, it will be converted to null. 
6. Remove `check_array_format()`, because it's logic is wrong and meaningless
7. Add stream load csv test cases and more parquet broker load tests
2022-10-20 15:52:31 +08:00
9ac4cfc9bb [bugfix](array-type) ColumnDate lost is_date_type after cloned (#13420)
Problem:
IColumn::is_date property will lost after ColumnDate::clone called.

Fix:
After ColumnDate created, also set IColumn::is_date.

Co-authored-by: cambyzju <zhuxiaoli01@baidu.com>
2022-10-19 21:29:36 +08:00
49b060418a [optimization](array-type) array_min/array_max function support the date/datetime type (#13407)
This pr is used to expand the supported data type for array_min/array_max function.
Before the change , the array_min/array_max function can't support the date/datetime type.
After the change, array_min/array_max function can support the date/datetime type.
Co-authored-by: hucheng01 <hucheng01@baidu.com>
2022-10-17 23:38:20 +08:00
20b583c91e [Bug](array-type) Fix memory buffer overflow (#13074) 2022-10-10 11:42:13 +08:00
b8b18e5153 [enhancement](array-type) Handle cast empty string value to array (#13028)
Handle empty value between two comma when cast string to array type.

before:
mysql> select cast("[a,b,c,,,,]" as array<string>);
+-----------------------------------+
| CAST('[a,b,c,,,,]' AS ARRAY<TEXT>) |
+-----------------------------------+
| ['a', 'b', 'c', ',', ',']                |
+-----------------------------------+
1 row in set (0.01 sec)

after:
mysql> select cast("[a,b,c,,,,]" as array<string>);
+-----------------------------------+
| CAST('[a,b,c,,,,]' AS ARRAY<TEXT>) |
+-----------------------------------+
| ['a', 'b', 'c', '', '', '']                |
+-----------------------------------+
1 row in set (0.01 sec)
2022-10-08 21:45:42 +08:00
b083fb6d5f [fix](decimal) retain Decimal trailing zero when select on fe (#13065) 2022-10-04 21:31:18 +08:00
56b2fc43d4 [enhancement](array-type) shrink column suffix zero for type ARRAY<CHAR> (#12443)
In compute level, CHAR type will shrink suffix zeros.
To keep the logic the same as CHAR type, we also shrink for ARRAY or ARRAY<ARRAY> types.

Co-authored-by: cambyzju <zhuxiaoli01@baidu.com>
2022-09-13 23:24:48 +08:00
e7303c12c7 [Enhancement](array-type) Support Floating/Decimal type for array aggregation functions (#12271) 2022-09-03 09:55:56 +08:00
fc05d54f0d [fix](array-type) array_sort function with empty input #12175
Co-authored-by: cambyzju <zhuxiaoli01@baidu.com>
2022-09-01 10:54:09 +08:00
2715ff9e0f [Enhancement](select) Make select variables request handled in fe without be to avoid potential blocked problem when login (#12111) 2022-08-29 23:07:30 +08:00
2b3a5b5fdd [fix](array-type) add ARRAY_BOOLEAN support for lots of array functions #12079
Co-authored-by: cambyzju <zhuxiaoli01@baidu.com>
2022-08-26 18:00:29 +08:00
0a450d96a5 [Fix] fix cast(array<not_null> as array<>) causes be core dump (#11649) 2022-08-24 22:51:30 +08:00
ff1971f916 [improvement](test) add dryRun option and group all cases into either p0 or p1 (#11576)
1. add dryRun option to list tests
2. group all cases into p0 p1 p2
2022-08-17 22:45:53 +08:00