325a1d4b28
[vectorized](function) support array_count function ( #18557 )
...
support array_count function.
array_count：Returns the number of non-zero and non-null elements in the given array.
2023-05-16 17:00:01 +08:00
20395ce501
[feature](array_function): add support for array_cum_sum function ( #18231 )
2023-04-27 09:57:13 +08:00
925efc1902
[bug](map-type)fix some bugs in map and map element function ( #18935 )
...
fix some bugs in map and map element function.
2023-04-26 22:10:15 +08:00
e412dd12e8
[chore](build) Use include-what-you-use to optimize includes (PART II) ( #18761 )
...
Currently, there are some useless includes in the codebase. We can use a tool named include-what-you-use to optimize these includes. By using a strict include-what-you-use policy, we can get lots of benefits from it.
2023-04-19 23:11:48 +08:00
79c446c89f
[enhancement](exception) Column filter/replicate supports exception safety ( #18503 )
2023-04-18 19:23:09 +08:00
43392918cd
[Optimization](functions)Optimize function call for const columns. ( #18310 )
2023-04-12 11:11:01 +08:00
1238f6de97
[bug](array) fix be core in array_with_constant/array_repeat function when the first argument is nullable ( #18404 )
...
fix be core in array_with_constant/array_repeat function when the first argument is nullable
2023-04-11 19:46:41 +08:00
0517616242
[vectorized](function) support array_repeat function to be compatible with hive syntax ( #18028 )
...
---------
Co-authored-by: zhangyu209 <zhangyu209@meituan.com >
2023-04-08 15:50:28 +08:00
8b85c55117
[vectorized](function) Support array_shuffle and shuffle function. ( #18116 )
...
---------
Co-authored-by: zhangyu209 <zhangyu209@meituan.com >
2023-04-04 08:53:13 +08:00
d4688620e9
[opt](array) optimize array_sortby using qsort instead of bubble sort #18311
2023-04-03 17:10:51 +08:00
94e3472050
[bug](function) fix count equal function return incorrect value ( #18200 )
...
fix count equal function return incorrect value
2023-04-03 11:20:36 +08:00
20b3bdb000
[vectorized](function) support array_first_index function ( #18175 )
...
mysql> select array_first_index(x->x+1>3, [2, 3, 4]);
+-------------------------------------------------------------------+
| array_first_index(array_map([x] -> x(0) + 1 > 3, ARRAY(2, 3, 4))) |
+-------------------------------------------------------------------+
| 2 |
+-------------------------------------------------------------------+
mysql> select array_first_index(x -> x is null, [null, 1, 2]);
+----------------------------------------------------------------------+
| array_first_index(array_map([x] -> x(0) IS NULL, ARRAY(NULL, 1, 2))) |
+----------------------------------------------------------------------+
| 1 |
+----------------------------------------------------------------------+
mysql> select array_first_index(x->power(x,2)>10, [1, 2, 3, 4]);
+---------------------------------------------------------------------------------+
| array_first_index(array_map([x] -> power(x(0), 2.0) > 10.0, ARRAY(1, 2, 3, 4))) |
+---------------------------------------------------------------------------------+
| 4 |
+---------------------------------------------------------------------------------+
2023-03-31 12:51:29 +08:00
f800ba8f4c
[Exec](opt) Optimize function call for const columns ( #18212 )
2023-03-31 11:36:21 +08:00
ea41d94582
[Improve](complex-type) Support Count(complexType) ( #17868 )
...
Support count function for ARRAY/MAP/STRUCT type
2023-03-30 15:43:32 +08:00
525f15dddf
[vectorized](function) support array_sortby function ( #18071 )
2023-03-30 11:07:49 +08:00
115e52c16c
[Opt](array) optimize_array_sort ( #18123 )
2023-03-27 22:01:24 +08:00
45ad297a1d
[Enchancement](function) change aggregate function creator to return AggregateFunctionPtr ( #18025 )
...
change creator_type to return AggregateFunctionPtr.
remove some function and use creator directly.
2023-03-26 11:41:34 +08:00
360d3050bc
[Feature](array-function) Support array_reverse_sort function ( #17754 )
...
Co-authored-by: zhangyu209 <zhangyu209@meituan.com >
2023-03-25 21:58:11 +08:00
089a91ecd5
[vectorized](function) support array_exists lambda function ( #17931 )
...
Co-authored-by: zhangyu209 <zhangyu209@meituan.com >
2023-03-23 11:11:39 +08:00
4193884a32
[feature](array_zip) Support array_zip function ( #17696 )
2023-03-21 18:44:30 +08:00
e7e13bc338
[optimize](array function) array_apply fucntion vectorized compute column_filter loop ( #17687 )
2023-03-19 10:18:09 +08:00
c302fa2564
[Feature](array-function) Support array_pushfront function ( #17584 )
2023-03-13 14:26:02 +08:00
55c42da511
[Feature](array) Support array<decimalv3> data type ( #16640 )
2023-03-13 10:48:13 +08:00
e1bf9411de
[feature](array function) add support for array_enumerate_uniq ( #17541 )
...
add support for array_enumerate_uniq()
2023-03-10 10:20:49 +08:00
06dee69174
[Refactor](map) remove using column array in map to reduce offset column ( #17330 )
...
1. remove column array in map
2. add offsets column in map
Aim to reduce duplicate offset from key-array and value-array in disk
2023-03-09 11:22:26 +08:00
65b8dfc7ff
[Enchancement](function) Inline some aggregate function && remove nullable combinator ( #17328 )
...
1. Inline some aggregate function
2. remove nullable combinator
2023-03-09 10:39:04 +08:00
4ea0d6c5fa
[feature](array_function) add support for array_popfront ( #17416 )
2023-03-08 13:57:38 +08:00
b1d65f855d
[Feature](array-function) Support array_concat function ( #17436 )
2023-03-08 13:57:16 +08:00
06468ba627
[vectorized](bug) fix array constructor function change origin column from block ( #17296 )
2023-03-07 16:42:23 +08:00
7cb6c522b0
[Enhancement](array) vectorized string equal comparasion in array_contains function
...
use StringRef instead of string_view operator == for vectorized impl for array_contains function.
- test data: 10,000,000 rows with a ARRAY<STRING> column. There are 10 elements, average length 11 chars, in the array column in each row.
- test SQL: `select count() from test_like_array where array_contains(s_arr, 'xxxxxxxx');`
- test result: 0.76 sec vs. 0.52 sec, 30% time reduced
2023-02-26 19:42:26 +08:00
c4edea5936
[Enchancement](function) refact and optimize some function register ( #16955 )
...
refact and optimize some function register
2023-02-24 10:05:11 +08:00
526a66e9fb
[Function](array-type) support array_apply ( #17020 )
...
Filter array to match specific binary condition
```
mysql> select array_apply([1000000, 1000001, 1000002], '=', 1000002);
+-------------------------------------------------------------+
| array_apply(ARRAY(1000000, 1000001, 1000002), '=', 1000002) |
+-------------------------------------------------------------+
| [1000002] |
+-------------------------------------------------------------+
```
2023-02-23 17:38:16 +08:00
2bc014d83a
[Enchancement](function) remove unused params on aggregate function ( #16886 )
...
remove unused params on aggregate function
2023-02-20 11:08:45 +08:00
1b3902baa2
[Feature](Complex-type) Add struct and map type to Doris ( #16444 )
...
This commit support:
1、Insert + select for struct/map type
2、Json stream load for struct type
3、m[key] function for map type
How to use:
Set the fe config to create table for struct and map type
1、admin set frontend config("enable_struct_type" = "true");
2、admin set frontend config("enable_map_type" = "true");
#16547
Co-authored-by: xy720 <xuyang25@baidu.com >
Co-authored-by: amory <wangqiannan@selectdb.com >
Co-authored-by: cambyzju <zhuxiaoli01@baidu.com >
Co-authored-by: hucheng01 <hucheng01@baidu.com >
2023-02-10 11:00:33 +08:00
41947c73eb
[Feature](array-function) Support array functions for nested type datev2 and datetimev2 ( #16382 )
2023-02-08 12:51:07 +08:00
3e8b3658c7
[feature-wip](decimalv3) Support basic agg and arithmetic operations for decimal v3 ( #14513 )
2022-11-29 15:12:41 +08:00
7ae7830c50
[improvement](function)add size function alias array_size ( #14594 )
...
* add size function alias
* fix
2022-11-25 22:29:48 +08:00
7ba4cd764a
[enhancement](array-function) array_position,array_contains,countequal which in FunctionArrayIndex handle target NULL ( #14564 )
...
in the previous, the result is:
```
mysql> select array_position([1, null], null);
+--------------------------------------+
| array_position(ARRAY(1, NULL), NULL) |
+--------------------------------------+
| NULL |
+--------------------------------------+
1 row in set (0.02 sec)
```
but after this commit, the result become:
```
mysql> select array_position([1, null], null);
+--------------------------------------+
| array_position(ARRAY(1, NULL), NULL) |
+--------------------------------------+
| 2 |
+--------------------------------------+
1 row in set (0.02 sec)
```
2022-11-25 14:19:50 +08:00
bc699511d0
[Fix](array-function) fix array_distinct null values ( #14544 )
...
in the previous the result is:
```
mysql> select array_distinct([1,1,3,3,null, null, null]);
+-----------------------------------------------------+
| array_distinct(ARRAY(1, 1, 3, 3, NULL, NULL, NULL)) |
+-----------------------------------------------------+
| [1, 3, NULL, NULL, NULL] |
+-----------------------------------------------------+
1 row in set (0.00 sec)
```
after this fix, the result becomes:
```
mysql> select array_distinct([1,1,3,3,null, null, null]);
+-----------------------------------------------------+
| array_distinct(ARRAY(1, 1, 3, 3, NULL, NULL, NULL)) |
+-----------------------------------------------------+
| [1, 3, NULL] |
+-----------------------------------------------------+
1 row in set (0.00 sec)
```
2022-11-24 19:07:28 +08:00
1ec7f45fb6
[Bug](avg) Fix avg for bigint ( #14433 )
2022-11-22 10:29:59 +08:00
16d8a1853a
[Bug](array-function) array set function not handle all null value ( #14318 )
2022-11-22 09:07:43 +08:00
2c42f0a905
[refactor](decimalv3) Refine code for DecimalV3 ( #14394 )
2022-11-19 16:57:17 +08:00
b4aef889f2
[feature-array](array-function) add array constructor function array() ( #14250 )
...
* [feature-array](array-function) add array constructor function `array()`
```
mysql> select array(qid, creationDate) from nested_c_2 limit 10;
+------------------------------+
| array(`qid`, `creationDate`) |
+------------------------------+
| [1000038, 20090616074056] |
| [1000069, 20090616075005] |
| [1000130, 20090616080918] |
| [1000145, 20090616081545] |
+------------------------------+
10 rows in set (0.01 sec)
```
2022-11-19 10:49:50 +08:00
f86886f8f5
[Feature](function) Support array_compact function ( #14141 )
2022-11-15 14:24:37 +08:00
43490a33a5
[feature-array](array-type) Add array function array_with_constant ( #14115 )
...
Return array of constants with length num.
```
mysql> select array_with_constant(4, 1223);
+------------------------------+
| array_with_constant(4, 1223) |
+------------------------------+
| [1223, 1223, 1223, 1223] |
+------------------------------+
1 row in set (0.01 sec)
```
co-authored-by @eldenmoon
2022-11-11 22:08:43 +08:00
7ffe88b579
[feature-array](array-type) Add array function array_popback ( #13641 )
...
Remove the last element from array.
```
mysql> select array_popback(['test', NULL, 'value']);
+-----------------------------------------------------+
| array_popback(ARRAY('test', NULL, 'value')) |
+-----------------------------------------------------+
| [test, NULL] |
+-----------------------------------------------------+
```
2022-11-07 10:48:16 +08:00
5dd052d386
[Function](array) support array_range function ( #13547 )
...
* array_range with 3 impl
* [Function](array) support array_range function
* update
* update code
2022-10-28 08:40:24 +08:00
235c105554
[feature-array](array-type) Add array function array_enumerate ( #13612 )
...
Add array function array_enumerate
2022-10-25 15:12:11 +08:00
177e82bdab
[Enhancement](array-type) Add type derivation for array functions ( #13534 )
...
From now, we don't support type derivation for array function's arguments.
So that the cases below will return wrong values or even cause be core.
mysql> select array_union([1],[10000000]);
+----------------------------------------+
| array_union(ARRAY(1), ARRAY(10000000)) |
+----------------------------------------+
| [1, -128] |
+----------------------------------------+
1 row in set (0.03 sec)
mysql> select array_union([NULL],[1]);
ERROR 1105 (HY000): RpcException, msg: io.grpc.StatusRuntimeException: UNAVAILABLE: Network closed for unknown reason
mysql> select array_union([],[1]);
ERROR 1105 (HY000): RpcException, msg: io.grpc.StatusRuntimeException: UNAVAILABLE: Network closed for unknown reason
This commit make a small fix to derivate the argument types of the array function
1、 For null type in arguments, cast the null type to boolean type, because null type should not be seen in be.
2、For different types in arguments, cast all arguments type to their compatible type.
2022-10-24 11:51:47 +08:00
a555f45834
[fix](array-type) fix the wrong result of array_join function ( #13477 )
...
this pr is used to fix the wrong result of array_join function.
before the change, the array_join function will return wrong result.
MySQL [example_db]> select array_join(["", "1", "2"], '');
+--------------------------------------+
| array_join(ARRAY('', '1', '2'), '') |
+--------------------------------------+
| 1_2 |
+--------------------------------------+
3.after the change, the array_join function will return correct result.
MySQL [example_db]> select array_join(["", "1", "2"], '');
+--------------------------------------+
| array_join(ARRAY('', '1', '2'), '') |
+--------------------------------------+
| _1_2 |
+--------------------------------------+
Issue Number: #7570
2022-10-21 17:36:44 +08:00
