Commit Graph

436 Commits

Author SHA1 Message Date
e9052e2180 [cherry-pick](branch-21) fix mod function cause core dump (#37999) (#38308)
## Proposed changes
cherry-pick from master https://github.com/apache/doris/pull/37999

<!--Describe your changes.-->

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
2024-07-25 12:06:21 +08:00
d19b3a5cb6 [fix](function) the bucket number arg of width_bucket should be a positive integer value (#37892) (#38295)
## Proposed changes

pick #37892
2024-07-25 11:56:47 +08:00
79a6496bb6 [branch-2.1](function) fix wrong result when convert_tz is out of bound (#37358) (#38313)
## Proposed changes

pick https://github.com/apache/doris/pull/37358

before:
```sql
mysql> select CONVERT_TZ(cast('0000-01-01 00:00:00.00001'  as DATETIMEV1), cast('Asia/Shanghai' as VARCHAR(65533)), cast('America/Los_Angeles' as VARCHAR(65533)));
+---------------------------------------------------------------------------------------------------------------------------------------------------+
| convert_tz(cast('0000-01-01 00:00:00.00001' as DATETIME), cast('Asia/Shanghai' as VARCHAR(65533)), cast('America/Los_Angeles' as VARCHAR(65533))) |
+---------------------------------------------------------------------------------------------------------------------------------------------------+
| q535-12-31 08:01:19                                                                                                                               |
+---------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.12 sec)
```
now:
```sql
mysql> select CONVERT_TZ(cast('0000-01-01 00:00:00.00001'  as DATETIMEV1), cast('Asia/Shanghai' as VARCHAR(65533)), cast('America/Los_Angeles' as VARCHAR(65533)));
+---------------------------------------------------------------------------------------------------------------------------------------------------+
| convert_tz(cast('0000-01-01 00:00:00.00001' as DATETIME), cast('Asia/Shanghai' as VARCHAR(65533)), cast('America/Los_Angeles' as VARCHAR(65533))) |
+---------------------------------------------------------------------------------------------------------------------------------------------------+
| NULL                                                                                                                                              |
+---------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.09 sec)
```
2024-07-25 11:32:44 +08:00
2759383365 [branch-2.1](timezone) refactor tzdata load to accelerate and unify timezone parsing (#37062) (#37269)
pick https://github.com/apache/doris/pull/37062

1. revert https://github.com/apache/doris/pull/25097. we decide to rely
on OS. not maintain independent tzdata anymore to keep result
consistency
2. refactor timezone load. removed rwlock.

before:
```sql
mysql [optest]>select count(convert_tz(d, 'Asia/Shanghai', 'America/Los_Angeles')), count(convert_tz(dt, 'America/Los_Angeles', '+00:00')) from dates;
+-------------------------------------------------------------------------------------+--------------------------------------------------------+
| count(convert_tz(cast(d as DATETIMEV2(6)), 'Asia/Shanghai', 'America/Los_Angeles')) | count(convert_tz(dt, 'America/Los_Angeles', '+00:00')) |
+-------------------------------------------------------------------------------------+--------------------------------------------------------+
|                                                                            16000000 |                                               16000000 |
+-------------------------------------------------------------------------------------+--------------------------------------------------------+
1 row in set (6.88 sec)
```
now:
```sql
mysql [optest]>select count(convert_tz(d, 'Asia/Shanghai', 'America/Los_Angeles')), count(convert_tz(dt, 'America/Los_Angeles', '+00:00')) from dates;
+-------------------------------------------------------------------------------------+--------------------------------------------------------+
| count(convert_tz(cast(d as DATETIMEV2(6)), 'Asia/Shanghai', 'America/Los_Angeles')) | count(convert_tz(dt, 'America/Los_Angeles', '+00:00')) |
+-------------------------------------------------------------------------------------+--------------------------------------------------------+
|                                                                            16000000 |                                               16000000 |
+-------------------------------------------------------------------------------------+--------------------------------------------------------+
1 row in set (2.61 sec)
```
3. now don't support timezone offset format string like 'UTC+8', like we
already said in
https://doris.apache.org/docs/dev/query/query-variables/time-zone/#usage
4. support case-insensitive timezone parsing in nereids.
5. a bug when parse timezone using nereids. should check DST by input,
but wrongly by now before. now fixed.

doc pr: https://github.com/apache/doris-website/pull/810
2024-07-15 10:56:48 +08:00
8de13c5cc8 [fix](function) error scale set in unix_timestamp (#36110) (#37619)
## Proposed changes

```
mysql [test]>set DEBUG_SKIP_FOLD_CONSTANT = true;
Query OK, 0 rows affected (0.00 sec)

mysql [test]>select cast(unix_timestamp("2024-01-01",'yyyy-MM-dd') as bigint);
+------------------------------------------------------------+
| cast(unix_timestamp('2024-01-01', 'yyyy-MM-dd') as BIGINT) |
+------------------------------------------------------------+
|                                           1704038400000000 |
+------------------------------------------------------------+
```
now
```
mysql [test]>select cast(unix_timestamp("2024-01-01",'yyyy-MM-dd') as bigint);
+------------------------------------------------------------+
| cast(unix_timestamp('2024-01-01', 'yyyy-MM-dd') as BIGINT) |
+------------------------------------------------------------+
|                                                 1704038400 |
+------------------------------------------------------------+
1 row in set (0.01 sec)
```

The column does not have a scale set, but the cast uses the scale to
perform the cast.


<!--Describe your changes.-->

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
2024-07-15 10:00:04 +08:00
d237a4d303 [fix](array)fix array_except/union for left const return only one row result #36776 (#36986) 2024-06-30 12:25:17 +08:00
8fc70e32bc [fix](planner) fix wrong resut of function ifnull/coalesce caused by … (#36727) 2024-06-25 17:32:04 +08:00
4a117800ca [Bug](Function) fix json contains with empty value (#36320) (#36418) 2024-06-18 10:20:45 +08:00
dd567fa774 [fix](function) support return JsonType for If function (#35199)
add a FunctionSignature for If to support return Type is JsonType.
2024-05-24 16:23:58 +08:00
98b2bda660 [opt](Nereids) remove restrict for count(*) in window (#35220)
support count(*) used for window function

CREATE TABLE `t1` (
  `id` INT NULL,
  `dt` TEXT NULL
)
DISTRIBUTED BY HASH(`id`) BUCKETS 10
PROPERTIES (
"replication_allocation" = "tag.location.default: 1"
);

select *, count(*) over() from t1;
2024-05-24 16:23:58 +08:00
eb7eaee386 [fix](function) money format (#34680) 2024-05-18 18:35:29 +08:00
5719f6ff0c [fix](planner) fix date_xxx functions without complete function signature (#34761)
Problem:
When using current_date as input of functions like date_sub,
fold constant would failed cause of missing of function signature in Planner

Solved:
Add complete function signature of functions like date_sub
2024-05-18 18:26:38 +08:00
46bf43130f [test](case) error format case in test_query_json_object (#34722)
error format case in test_query_json_object
2024-05-18 18:07:23 +08:00
876248aa4e [fix](function) json_object can not input null value (#34591) 2024-05-18 18:00:48 +08:00
ca9eb56233 [Fix](functions) fix strcmp return value #34565 2024-05-12 09:49:38 +08:00
c742a327e6 [Fix](Planner) add date/month/year add function fe constant fold with dateV2/dateTimeV2 (#34483)
Problem and Cause:
In original planner, date_add function would choose different data type of datetime or datev2. Which when original planner choose datev2 as constant date type. Fe could not folding date_add function because missing of function signature of folding constant date_add(datev2, int)

Solved:
Add corresponding function signatures of date_add/months_add/years_add in original planner
2024-05-11 15:04:35 +08:00
58c19e33b3 [fix](round) Fix incorrect decimal scale inference in round functions (#34471)
* FIX NEEDED

* FORMAT

* FORMAT

* FIX TEST
2024-05-11 11:42:12 +08:00
e38801968d [Fix](functions) Fix bug in makedate and str_to_date functions 2024-05-10 22:14:25 +08:00
845732b440 [WIP](test) remove enable_nereids_planner in regression cases (part 3) (#34558)
previous PR:
part 1: #34417
part 2: #34490
2024-05-10 22:11:01 +08:00
9b712b03b4 [FIX]fix is_ip_address_in_range func with const param (#34266) 2024-05-10 14:37:20 +08:00
956ae2f83d [opt](Nereids) let behavior of function char same with legacy planner (#34415)
1. first argument must be string like literal
2. only support utf-8 charset
2024-05-07 07:34:34 +08:00
f7900b53ce [enhancement](function) floor/ceil/round/round_bankers can use column as scale argument (#34391) 2024-05-06 22:18:36 +08:00
996222c30e [fix](test) let test_ntile_function happy with Nereids (#34294) 2024-04-29 20:59:48 +08:00
20bd0c2987 [FIX](cases )fix ipv6 value for regress case 2024-04-29 13:37:29 +08:00
8d98c71079 [FIX]fix cidr func with const param (#33968) 2024-04-24 17:13:50 +08:00
2f60dcf890 [test](hll) fix unstable case without order by clause (#33947) 2024-04-24 17:13:50 +08:00
22a6b1d3f5 [feature](function) support hll functions hll_from_base64, hll_to_base64 (#32089)
Issue Number: #31320 

Support two hll functions:

- hll_from_base64
Convert a base64 string(result of function hll_to_base64) into a hll.
- hll_to_base64
Convert an input hll to a base64 string.
2024-04-17 23:42:13 +08:00
f35dd3fc35 [chore](test) let some case suitable for legacy planner and nereids (#33352) 2024-04-11 09:31:50 +08:00
bf022f9d8d [enhancement](function truncate) truncate can use column as scale argument (#32746)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-04-10 14:53:56 +08:00
02b24abed2 [Fix](Nereids) ntile function should check argument (#32994)
Problem:
when ntile using 0 as parameter, be would core because no checking of parameter
Solved:
check parameter in fe analyze
2024-04-10 14:53:56 +08:00
121a0901c8 [chore](test) let suite compatible with Nereids (#32964) 2024-04-10 11:34:29 +08:00
2a0644f442 [Fix](function) Fix unix_timestamp core for string input (#32871) 2024-04-09 12:48:35 +08:00
39b768b9ea [branch-2.1](thirdparty) upgrade arrow to 15.0.2 #32827 2024-03-27 08:35:40 +08:00
8b960beaec [refactor](nereids)unify outputTupleDesc and projection (#32093)
* unify join node output project
2024-03-24 08:05:42 +08:00
8e3d28b93a [Feature] Support Array Type compare function for nereids planner (#31701)
Support Array Type compare function for nereids planner
2024-03-21 14:07:49 +08:00
715eed0748 [opt](like) opt LIKE and REGEXP clause with concat(col, pattern_str) (#32333)
opt LIKE and REGEXP clause with concat(col, pattern_str)
2024-03-21 14:07:24 +08:00
680fce8825 [chore](test) let regression test work well with nereids (#32199) 2024-03-15 18:02:01 +08:00
ccd21a6ea4 [Improve](InPredict) enhance in predict with array type (#31828) 2024-03-12 14:19:14 +08:00
e91d16854b [fix](function) fix date_format function execution error on fe (#31645) 2024-03-07 16:53:19 +08:00
7f3a666fac [fix](planner) fix ifnull and nvl function with one parameters exception message anbugious (#31808)
When ifnull or nvl funtion have only one parameter, nereids planner would throw an exception and go back to
original planner/ Original planner get secend parameter directly without check, so it return unexpected error message
2024-03-06 13:08:30 +08:00
2d6e975d5a [fix](cast) fix wrong result while cast string to float (#31781)
Issue Number: close #31518
2024-03-06 13:07:59 +08:00
7998da4691 [fix](cast) wrong result while cast const to double then to string (#31657)
Issue Number: close #31514
2024-03-06 13:06:27 +08:00
ff3c7765d6 [Bug](Func) Fix negative number in bitmap return error result (#31563) 2024-02-29 12:38:03 +08:00
f163d56a98 [feature](function) support sequence function(alias of array_range), enhance both to handle datetimev2 (#30823) 2024-02-27 10:12:19 +08:00
3cee6c6722 [fix](function) fix unexpected be core in string search function (#31312)
Fix be core in multi_match_any/multi_search_all_positions functions.
2024-02-27 10:12:18 +08:00
8f77e6363a [Feature](function) Support xxhash function like murmur hash function (#31193) 2024-02-23 19:03:28 +08:00
2af920c6f8 [chore](regression) Update q03.sql (#31063)
add 'order by' of case
2024-02-20 16:24:05 +08:00
6cf7468073 [enhancement](function) change some function nullable mode (#30991)
change some function nullable mode
2024-02-18 14:45:25 +08:00
e68019c10a [Function](Exec) Support windows function cume_dist (#30997) 2024-02-16 10:16:40 +08:00
40e1326bc9 [feature](window-func) support percent_rank window function (#30926) 2024-02-16 10:12:25 +08:00