Commit Graph

72 Commits

Author SHA1 Message Date
ebc178af23 [fix](nereids)acos function should return null literal instead of NaN value (#37996)
pick from master https://github.com/apache/doris/pull/37932

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
2024-07-18 09:28:56 +08:00
0cff539810 [feature](function) support new function replace_empty (#36283) (#36656)
#36283
2024-06-21 16:46:22 +08:00
f751ca4e04 [branch-2.1](functions) fix be crash for function random_bytes and mark_first/last_n (#36003)
pick #35884
2024-06-07 10:30:41 +08:00
fe7d2b8159 [Fix](nereids) ignore slot implements SlotNotFromChildren when check the slot from children in NormalizeAggregate (#34171) 2024-05-07 07:48:05 +08:00
22a6b1d3f5 [feature](function) support hll functions hll_from_base64, hll_to_base64 (#32089)
Issue Number: #31320 

Support two hll functions:

- hll_from_base64
Convert a base64 string(result of function hll_to_base64) into a hll.
- hll_to_base64
Convert an input hll to a base64 string.
2024-04-17 23:42:13 +08:00
2746ee9d3a [FIX](regresstest) fix array_range case for regress test (#33337) 2024-04-17 23:42:00 +08:00
7f3a666fac [fix](planner) fix ifnull and nvl function with one parameters exception message anbugious (#31808)
When ifnull or nvl funtion have only one parameter, nereids planner would throw an exception and go back to
original planner/ Original planner get secend parameter directly without check, so it return unexpected error message
2024-03-06 13:08:30 +08:00
9c4708ee74 [function](random_bytes)add random_bytes function (#31547)
SELECT random_bytes(10);

random_bytes(10) |
----------------------+
0x9b8ea00b7d1084bc5b26|
2024-02-29 16:44:39 +08:00
f163d56a98 [feature](function) support sequence function(alias of array_range), enhance both to handle datetimev2 (#30823) 2024-02-27 10:12:19 +08:00
1ed24117ac [function](url_decode)add url_decode function (#30667) 2024-02-05 22:23:00 +08:00
6442663735 [Function](exec) upport atan2 math function (#30672)
Co-authored-by: Rohit Satardekar <rohitrs1983@gmail.com>
2024-02-04 14:28:38 +08:00
19f57b544e support cosh math function (#30602)
Co-authored-by: Rohit Satardekar <rohitrs1983@gmail.com>
2024-01-31 23:53:39 +08:00
8b61b7c6cd [exec](function) Add tanh func (#30555) 2024-01-31 23:53:39 +08:00
c46fa339d0 [feature](Nereids): make count() avaliable for all data type (#26180)
make count() avaliable for all data type
2023-11-02 17:20:24 +08:00
e77b98be88 [fix](months_diff) fix wrong result of months_diff (#25577) 2023-10-19 14:29:47 +08:00
af8832389f [feature](Nereids) add 4 array functions (#25488)
- array_concat
- array_pushback
- array_pushfront
- array_zip
2023-10-17 04:45:15 -05:00
42f8b253aa [function](nereids) support array_apply/array_repeat/group_uniq_array/ipv4numtostring (#25249)
nereids support functions: array_apply/array_repeat/group_uniq_array/ipv4numtostring
2023-10-12 11:08:42 +08:00
b91bce8a62 [feature](Nereids) add array distance functions (#25196)
- l1_distance
- l2_distance
- cosine_distance
- inner_product
2023-10-10 21:35:06 -05:00
37247ac449 [opt](Nereids) add two args signature to trim family functions (#25169) 2023-10-09 07:17:52 -05:00
320709b9ff [opt](Nereids) support like and regexp function (#25148) 2023-10-09 02:55:57 -05:00
3a45001447 [fix](Nerids) fix error when the view has lambda functions (#25067)
1. To ensure compatibility with the original optimizer, expose the non-lambda signature of highorder function externally.
2. fix some bugs in toSql function in the original optimizer
2023-10-08 15:45:24 +08:00
5d138b6928 [remove](function) make execute_impl const and remove running_difference function (#24935) 2023-09-27 18:17:28 +08:00
90c5461ad2 [fix](Nereids) let dml work well (#24748)
Co-authored-by: sohardforaname <organic_chemistry@foxmail.com>

TODO:
1. support agg_state type
2. support implicit cast literal exception
3. use nereids execute dml for these regression cases:

- test_agg_state_nereids (for TODO 1)
- test_array_insert_overflow (for TODO 2)
- nereids_p0/json_p0/test_json_load_and_function (for TODO 2)
- nereids_p0/json_p0/test_json_unique_load_and_function (for TODO 2)
- nereids_p0/jsonb_p0/test_jsonb_load_and_function (for TODO 2)
- nereids_p0/jsonb_p0/test_jsonb_unique_load_and_function (for TODO 2)
- json_p0/test_json_load_and_function (for TODO 2)
- json_p0/test_json_unique_load_and_function (for TODO 2)
- jsonb_p0/test_jsonb_load_and_function (for TODO 2)
- jsonb_p0/test_jsonb_unique_load_and_function (for TODO 2)
- test_multi_partition_key (for TODO 2)
2023-09-26 21:08:24 +08:00
12bee54cf7 [fix](Nereids) function implict cast for json is not work well (#24852)
for sql like:

SELECT JSONB_EXTRACT('{"k1":"v31","k2":300}','$.k1');

the result should be

+------------------------------------------------+
| jsonb_extract('{"k1":"v31","k2":300}', '$.k1') |
+------------------------------------------------+
| "v31"                                          |
+------------------------------------------------+

but curent result is

+------------------------------------------------+
| jsonb_extract('{"k1":"v31","k2":300}', '$.k1') |
+------------------------------------------------+
| <null>                                         |
+------------------------------------------------+
2023-09-25 21:10:04 +08:00
22616d125d [function](bitmap) add function alias bitmap_andnot and bitmap_andnot_count (#24771) 2023-09-22 12:18:31 +08:00
5b590bbfcf [feat](Nereids) add lambda func array_last and array_first (#24682) 2023-09-21 12:23:34 +08:00
a3361df7b9 [Feat](Nereids) support json and jsonb datatype (#24156)
Feature:
support jsonb and json type in nereids

Document:
this feature supports these two datatype in nereids optimizer like original planner, the sql reference is same as before
[JSON - Apache Doris](https://doris.apache.org/zh-CN/docs/dev/sql-manual/sql-reference/Data-Types/JSON)
2023-09-20 14:32:22 +08:00
c3bd2a22d4 [feature](Nereids) add many array functions (#24301)
Add function array_filter, array_sortby, array_last_index, array_first_index, array_orderby, array_count
2023-09-19 18:58:49 +08:00
88adab3114 [fix](Nereids): fix be core when array_map is not nullable (#24488)
fix be core when array_map is not nullable
2023-09-16 20:39:15 +08:00
4fbb25bc55 [Enhancement](function) Support date_trunc(date) and use it in auto partition (#24341)
Support date_trunc(date) and use it in auto partition
2023-09-14 16:53:09 +08:00
dc28878f0e [FIX](function) fix size function for array map (#23920)
Issue Number: close #xxx
now we use select size(map(1, 2)); which will make be core
and we can make size function handle array & map column both
2023-09-06 14:32:06 +08:00
a136836770 [feature](Nereids) add two functions: char and covert (#23104)
add [char](https://doris.apache.org/zh-CN/docs/dev/sql-manual/sql-functions/string-functions/char/?_highlight=char) func
```
mysql> select char(68, 111, 114, 105, 115);
+--------------------------------------+
| char('utf8', 68, 111, 114, 105, 115) |
+--------------------------------------+
| Doris                                |
+--------------------------------------+
```

convert func
```
MySQL root@127.0.0.1:(none)> select convert(1 using gbk);
+-------------------+
| convert(1, 'gbk') |
+-------------------+
| 1                 |
+-------------------+
```
2023-08-30 17:09:06 +08:00
f7d2c1faf6 [feature](Nereids) support select key encryptKey (#23257)
Add select key

```
- CREATE ENCRYPTKEY key_name AS "key_string"
- select key my_key
+-----------------------------+
| encryptKeyRef('', 'my_key') |
+-----------------------------+
| ABCD123456789               |
+-----------------------------+
```
2023-08-28 14:07:26 +08:00
e84989fb6d [feature](Nereids) support map type (#23493) 2023-08-28 11:31:44 +08:00
0ccb7262a7 [feature](Nereids) add password func (#23244)
add password function
```
select password("123");
+-------------------------------------------+
| password('123')                           |
+-------------------------------------------+
| *23AE809DDACAF96AF0FD78ED04B6A265E05AA257 |
+-------------------------------------------+
```
2023-08-25 14:04:49 +08:00
3786ffec51 [opt](Nereids) add some array functions (#23324)
1. rename TVFProperties to Properties
2. add generating function explode and explode_outer
3. fix concat_ws could not apply on array
4. check tokenize second argument format on FE
5. add test case for concat_ws, tokenize, explode, explode_outer and split_by_string
2023-08-25 11:01:50 +08:00
da2eb69eba [test](Nereids) add array scalar function test cases (#23303) 2023-08-22 15:05:28 +08:00
d3b50e3b2a [BUG](date_trunc) fix date_trunc function only handle lower string (#22602)
fix date_trunc function only handle lower string
2023-08-05 12:53:13 +08:00
ec1ab1a3d2 [Improve](GEO)wkb input and output are represented as hexadecimal strings And delete EWKB (#18721) 2023-04-21 15:11:18 +08:00
f2d75cb492 [fix](Nereids) fix signature precision round for decimalv3 (#18639)
add decimalv3 signature to below functions:
ceil
dceil
dfloor
dround
floor
round
round_bankers
truncate
fix ComputePrecisionForRound to get correct signature
2023-04-14 18:18:41 +08:00
4d18ea30f4 [fix](Nereids) get_json_bigint should return bigint type (#18626) 2023-04-14 14:01:44 +08:00
2f64a8b387 [feature](GEO)Support read/write WKB/EWKB to gis types (#18526)
Support mutual conversion from wkb and gis types.also compatible with EWKB format
https://cwiki.apache.org/confluence/display/DORIS/DSIP-033%3A+More+GEO+functions
2023-04-13 16:25:18 +08:00
961f5d1bb7 [feature](function)Add St_Angle/St_Azimuth function (#18293)
Add St_Angle/St_azimuth function:
St_Angle:
Enter three point, which represent two intersecting lines. Returns the angle between these lines. Point 2 and point 1 represent the first line and point 2 and point 3 represent the second line. The angle between these lines is in radians, in the range [0, 2pi). The angle is measured clockwise from the first line to the second line.

`

mysql> SELECT ST_Angle(ST_Point(1, 0),ST_Point(0, 0),ST_Point(0, 1));
+----------------------------------------------------------------------+
| st_angle(st_point(1.0, 0.0), st_point(0.0, 0.0), st_point(0.0, 1.0)) |
+----------------------------------------------------------------------+
| 4.71238898038469 |
+----------------------------------------------------------------------+
1 row in set (0.04 sec)
`

St_azimuth:
Enter two point, and returns the azimuth of the line segment formed by points 1 and 2. The azimuth is the angle in radians measured between the line from point 1 facing true North to the line segment from point 1 to point 2.
`

mysql> SELECT st_azimuth(ST_Point(0, 0),ST_Point(1, 0));
+----------------------------------------------------+
| st_azimuth(st_point(0.0, 0.0), st_point(1.0, 0.0)) |
+----------------------------------------------------+
| 1.5707963267948966 |
+----------------------------------------------------+
1 row in set (0.04 sec)
2023-04-03 13:01:59 +08:00
6964d9f99c [fix](function) resubmit-fix AES/SM3/SM4 encrypt/ decrypt algorithm initialization vector bug (#17907)
* Revert "[fix](function) fix AES/SM3/SM4 encrypt/ decrypt algorithm initialization vector bug (#17420)"

This reverts commit 397cc011c4f1ba5a25c770258c13f1cd3f28b47d.

* [fix-resubmit](function) fix AES/SM3/SM4 encrypt/ decrypt algorithm initialization vector bug (#17420)

ECB algorithm, block_encryption_mode does not take effect, it only takes effect when init vector is provided.
Solved: 192/256 supports calculation without init vector

For other algorithms, an error should be reported when there is no init vector

Initialization Vector. The default value for the block_encryption_mode system variable is aes-128-ecb, or ECB mode, which does not require an initialization vector. The alternative permitted block encryption modes CBC, CFB1, CFB8, CFB128, and OFB all require an initialization vector.

Reference: https://dev.mysql.com/doc/refman/8.0/en/encryption-functions.html#function_aes-decrypt

Note: This fix does not support smooth upgrades. during upgrade process, query may report error: funciton not found
2023-03-29 21:13:01 +08:00
012f7bd031 [feature](function)Add ST_Area function (#18138) 2023-03-28 19:36:09 +08:00
bcf95cd920 [feature](function)Add ST_Angle_Sphere function (#17919) 2023-03-27 10:14:46 +08:00
ebef0c038d Revert "[fix](function) fix AES/SM3/SM4 encrypt/ decrypt algorithm initialization vector bug (#17420)" (#17887)
This reverts commit 397cc011c4f1ba5a25c770258c13f1cd3f28b47d.
2023-03-22 13:28:25 +08:00
365c8eed7e [fix](function) width_bucket should get min and max from each tuple (#17466) 2023-03-10 13:14:12 +08:00
397cc011c4 [fix](function) fix AES/SM3/SM4 encrypt/ decrypt algorithm initialization vector bug (#17420)
ECB algorithm, block_encryption_mode does not take effect, it only takes effect when init vector is provided.
Solved: 192/256 supports calculation without init vector

For other algorithms, an error should be reported when there is no init vector

Initialization Vector. The default value for the block_encryption_mode system variable is aes-128-ecb, or ECB mode, which does not require an initialization vector. The alternative permitted block encryption modes CBC, CFB1, CFB8, CFB128, and OFB all require an initialization vector.

Reference: https://dev.mysql.com/doc/refman/8.0/en/encryption-functions.html#function_aes-decrypt

Note: This fix does not support smooth upgrades. during upgrade process, query may report error: funciton not found
2023-03-09 09:51:41 +08:00
9aecd517b0 [test](Nereids) turn on all test in scalar function w (#17269)
turn on all test case in scalar function W except width_bucket(fix be bug in next PR)
turn off all test case for group_concat(distinct order by)
fix return nullable in TimestampArithmetic
2023-03-04 08:23:50 +08:00