database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
10,501 Commits 1 Branch 0 Tags
d784c99360f905d38a13ee5c8192e596d7c8ea7b
Commit Graph

306 Commits

Author SHA1 Message Date
Pxl
d784c99360 [Bug](planner) fix unassigned conjunct assigned on wrong node (#19672) 
* fix unassigned conjunct assigned on wrong node
 2023-05-17 10:28:22 +08:00
Pxl
7f73749b88 [Bug](pipeline) fix distributionColumnIds not updated correct when outputColumnUnique… (#19704) 
fix distributionColumnIds not updated correct when outputColumnUnique
 2023-05-17 00:13:10 +08:00
Ziyu Wang
325a1d4b28 [vectorized](function) support array_count function (#18557) 
support array_count function.
array_count：Returns the number of non-zero and non-null elements in the given array.
 2023-05-16 17:00:01 +08:00
Zhengguo Yang
6748ae4a57 [Feature] Collect the information statistics of the query hit (#18805) 
1. Show the query hit statistics for `baseall`

   ```sql
    MySQL [test_query_db]> show query stats from baseall;
    +-------+------------+-------------+
    | Field | QueryCount | FilterCount |
    +-------+------------+-------------+
    | k0    | 0          | 0           |
    | k1    | 0          | 0           |
    | k2    | 0          | 0           |
    | k3    | 0          | 0           |
    | k4    | 0          | 0           |
    | k5    | 0          | 0           |
    | k6    | 0          | 0           |
    | k10   | 0          | 0           |
    | k11   | 0          | 0           |
    | k7    | 0          | 0           |
    | k8    | 0          | 0           |
    | k9    | 0          | 0           |
    | k12   | 0          | 0           |
    | k13   | 0          | 0           |
    +-------+------------+-------------+
    14 rows in set (0.002 sec)

    MySQL [test_query_db]> select k0, k1,k2, sum(k3) from baseall  where k9 > 1 group by k0,k1,k2;
    +------+------+--------+-------------+
    | k0   | k1   | k2     | sum(`k3`)   |
    +------+------+--------+-------------+
    |    0 |    6 |  32767 |        3021 |
    |    1 |   12 |  32767 | -2147483647 |
    |    0 |    3 |   1989 |        1002 |
    |    0 |    7 | -32767 |        1002 |
    |    1 |    8 |    255 |  2147483647 |
    |    1 |    9 |   1991 | -2147483647 |
    |    1 |   11 |   1989 |       25699 |
    |    1 |   13 | -32767 |  2147483647 |
    |    1 |   14 |    255 |         103 |
    |    0 |    1 |   1989 |        1001 |
    |    0 |    2 |   1986 |        1001 |
    |    1 |   15 |   1992 |        3021 |
    +------+------+--------+-------------+
    12 rows in set (0.050 sec)

    MySQL [test_query_db]> show query stats from baseall;
    +-------+------------+-------------+
    | Field | QueryCount | FilterCount |
    +-------+------------+-------------+
    | k0    | 1          | 0           |
    | k1    | 1          | 0           |
    | k2    | 1          | 0           |
    | k3    | 1          | 0           |
    | k4    | 0          | 0           |
    | k5    | 0          | 0           |
    | k6    | 0          | 0           |
    | k10   | 0          | 0           |
    | k11   | 0          | 0           |
    | k7    | 0          | 0           |
    | k8    | 0          | 0           |
    | k9    | 1          | 1           |
    | k12   | 0          | 0           |
    | k13   | 0          | 0           |
    +-------+------------+-------------+
    14 rows in set (0.001 sec)
   ```

2. Show the query hit statistics summary for all the mv in a table

   ```sql
   MySQL [test_query_db]> show query stats from baseall all;
    +-----------+------------+
    | IndexName | QueryCount |
    +-----------+------------+
    | baseall   | 1          |
    +-----------+------------+
    1 row in set (0.005 sec)
   ```

3. Show the query hit statistics detail info for all the mv in a table

   ```sql
    MySQL [test_query_db]> show query stats from baseall all verbose;
    +-----------+-------+------------+-------------+
    | IndexName | Field | QueryCount | FilterCount |
    +-----------+-------+------------+-------------+
    | baseall   | k0    | 1          | 0           |
    |           | k1    | 1          | 0           |
    |           | k2    | 1          | 0           |
    |           | k3    | 1          | 0           |
    |           | k4    | 0          | 0           |
    |           | k5    | 0          | 0           |
    |           | k6    | 0          | 0           |
    |           | k10   | 0          | 0           |
    |           | k11   | 0          | 0           |
    |           | k7    | 0          | 0           |
    |           | k8    | 0          | 0           |
    |           | k9    | 1          | 1           |
    |           | k12   | 0          | 0           |
    |           | k13   | 0          | 0           |
    +-----------+-------+------------+-------------+
    14 rows in set (0.017 sec)
   ```

4. Show the query hit for a database

   ```sql
    MySQL [test_query_db]> show query stats for test_query_db;
    +----------------------------+------------+
    | TableName                  | QueryCount |
    +----------------------------+------------+
    | compaction_tbl             | 0          |
    | bigtable                   | 0          |
    | empty                      | 0          |
    | tempbaseall                | 0          |
    | test                       | 0          |
    | test_data_type             | 0          |
    | test_string_function_field | 0          |
    | baseall                    | 1          |
    | nullable                   | 0          |
    +----------------------------+------------+
    9 rows in set (0.005 sec)
   ```

5. Show query hit statistics for all the databases

   ```sql
    MySQL [(none)]> show query stats;
    +-----------------+------------+
    | Database        | QueryCount |
    +-----------------+------------+
    | test_query_db   | 1          |
    +-----------------+------------+
    1 rows in set (0.005 sec)
   ```
 2023-05-15 10:56:34 +08:00
starocean999
e9392780a9 [fix](nereids)fix some nereids planner bugs (#19509) 
1.some encrypt and decrypt functions have wrong blockEncryptionMode
2.topN node should compare tuples from intermediate_row_desc with first_sort_slot.tuple_id
3.must keep the limit if it's an uncorrelated in-subquery with limit on sort, like select a from t1 where a in ( select b from t2 order by xx limit yy )
 2023-05-12 09:06:16 +08:00
xy720
39ec8aa64c [refactor](complex-type) refactor array/map/struct literal to not invoke execute() function in prepare state (#19068) 2023-05-11 18:44:37 +08:00
herry2038
834bf2eab7 [feature](array) Add array_last lambda function (#18388) 
Add array_last lambda function
 2023-05-11 13:15:54 +08:00
Jerry Hu
47edc5a06e [fix](functions) Support nullable column for multi_string functions (#19498) 2023-05-11 01:13:13 +08:00
Pxl
5473795a51 [Bug](scan) forbiden push down in predicate when in_state->use_set is false (#19471) 
forbiden push down in predicate when in_state->use_set is false
 2023-05-10 11:12:20 +08:00
Gabriel
4c6ca88088 Revert "[refactor](function) ignore DST for function from_unixtime (#19151)" (#19333) 
This reverts commit 9dd6c8f87b73db238bfd38fb1d76f3796910f398.
 2023-05-06 16:33:58 +08:00
Gabriel
9dd6c8f87b [refactor](function) ignore DST for function from_unixtime (#19151) 2023-05-05 11:51:49 +08:00
yiguolei
8eab20d3df [bugfix](low cardinality) cached code is wrong will result wrong query result when many null pages (#19221) 
Sometimes the dict is not initialized when run comparison predicate here, for example, the full page is null, then the reader will skip read, so that the dictionary is not inited. The cached code is wrong during this case, because the following page maybe not null, and the dict should have items in the future.
This will result the dict string column query return wrong result, if there are many null values in the column.
I also add some regression test for dict column's equal query, larger than query, less than query.

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
 2023-04-29 21:28:41 +08:00
brody715
20395ce501 [feature](array_function): add support for array_cum_sum function (#18231) 2023-04-27 09:57:13 +08:00
TengJianPing
1afa7c786f [test](regression) add test case for bucket shuffle of datetime column (#19088) 2023-04-27 09:05:32 +08:00
zhangstar333
d037938a4c [vectorzied](function) fix year_floor get result is incorrectly (#19006) 2023-04-26 11:39:22 +08:00
herry2038
17b59df8dd [fix](function) Array_map compared offset rows one by one (#18406) 
Array_map 's multi columns compare not only nested data rows to be equal,but also the offsets data must equal each other.
 2023-04-25 19:12:19 +08:00
ZhangYu0123
93c48f2bb0 [fix](regression) fix show create table in_memory = false test result error #19022 2023-04-25 09:04:59 +08:00
xueweizhang
efebb3d21e [fix](schema) fix show create table get wrong random distribution info (#18895) 
* [fix](schema) fix show create table get wrong random distribution info


---------

Signed-off-by: nextdreamblue <zxw520blue1@163.com>
 2023-04-24 23:33:42 +08:00
Mellorsssss
ab2a6864bc [function](json) Json unquote (#18037) 2023-04-24 10:33:29 +08:00
Qi Chen
a9ac930e5f [Fix](mutli-catalogs) Fix jdbc regression tests. (#18927) 
- Fix `test_show_where` result.
- Remove `enable_decimal_conversion = true` in `test_mysql_jdbc_catalog`.
- Remove `test_show_create_catalog`.
 2023-04-23 19:42:13 +08:00
TengJianPing
b75f4c97f3 [function](string) support char function (#18878) 
* [function](string) support char function

* fix
 2023-04-22 08:36:48 +08:00
xueweizhang
f7651d8dfb (fix)[olap] not support in_memory=true now (#18731) 
* (fix)[olap] can not set in_memory=true now

---------

Signed-off-by: nextdreamblue <zxw520blue1@163.com>
 2023-04-21 21:55:37 +08:00
Liqf
ec1ab1a3d2 [Improve](GEO)wkb input and output are represented as hexadecimal strings And delete EWKB (#18721) 2023-04-21 15:11:18 +08:00
TengJianPing
0b074ade02 [fix](const column) fix coredump caused by const column for some functions (#18737) 2023-04-18 13:57:55 +08:00
zhangstar333
6b351a2818 [vectorzied](function) fix array_map function analyzed failed with order by clause (#18676) 
* [vectorzied](function) fix array_map function analyzed failed with order by clause

* add test
 2023-04-18 12:01:44 +08:00
luozenglin
1e06763366 [fix](bitmap) fix bitmap_count errors to set nullable to non-nullable bitmap col (#18689) 2023-04-17 13:23:27 +08:00
Gabriel
5300b21db7 [Bug](DECIMALV3) report failure if a decimal value is overflow (#18336) 2023-04-17 13:18:14 +08:00
zclllyybb
092d81f88a [BugFix](functions) fix multi_search_all_positions #18682 2023-04-17 08:32:57 +08:00
Liqf
8751f08d5a [bugfix](GEO)fix precision problem (#18642) 2023-04-14 10:39:19 +08:00
Liqf
2f64a8b387 [feature](GEO)Support read/write WKB/EWKB to gis types (#18526) 
Support mutual conversion from wkb and gis types.also compatible with EWKB format
https://cwiki.apache.org/confluence/display/DORIS/DSIP-033%3A+More+GEO+functions
 2023-04-13 16:25:18 +08:00
zhangstar333
df0aaece1d [Function](test) add some test cases for agg functions (#18610) 2023-04-13 10:23:41 +08:00
xy720
d57371da13 [feature](struct-type) support basic struct constructor function (#18190) 
This commit will support struct and named_struct function.
 2023-04-13 09:18:00 +08:00
xy720
1238f6de97 [bug](array) fix be core in array_with_constant/array_repeat function when the first argument is nullable (#18404) 
fix be core in array_with_constant/array_repeat function when the first argument is nullable
 2023-04-11 19:46:41 +08:00
Gabriel
101737023c [Bug](round) fix wrong scale for round-like function (#18507) 2023-04-11 09:36:59 +08:00
caoliang-web
8d4a7325a9 [regression](join)Add subquery join case to regression test #17865 2023-04-09 18:15:37 +08:00
WenYao
8a4a92f658 [Enchancement](compatible) show dateV2/datetimeV2 to date/datetime (#18358) 
show dateV2/datetimeV2 to date/datetime

modify show create table
modify desc table
use desc table all to get real type from column ColumnType
 2023-04-09 10:34:14 +08:00
ZhangYu0123
0517616242 [vectorized](function) support array_repeat function to be compatible with hive syntax (#18028) 
---------

Co-authored-by: zhangyu209 <zhangyu209@meituan.com>
 2023-04-08 15:50:28 +08:00
Gabriel
d881d71cd1 [Bug](cast) Fix bug for cast function between datetimev2 and string (#18442) 
Fix bug for cast function between datetimev2 and string
 2023-04-07 22:02:15 +08:00
HappenLee
c32adba1cf [Refactor](Pipeline) Refactor pipeline code to improve coverage (#18376) 
Refactor pipeline code to improve coverage
 2023-04-07 13:09:44 +08:00
TengJianPing
4e1cdb9ce7 [fix](agg_sort)fix bug of agg sort group concat with order by(#18447) 2023-04-07 08:42:36 +08:00
Tiewei Fang
759f1da32e [Enhencement](Backends) add HostName filed in backends table and delete backends table in information_schema (#18156) 
1.  Add `HostName` field for `show backends` statement and `backends()` tvf.
2. delete the `backends` table in `information_schema` database
 2023-04-07 08:30:42 +08:00
Gabriel
550c8aa648 [Bug](DECIMALV3) fix wrong decimal scale returned by function round (#18375) 2023-04-06 14:44:21 +08:00
morrySnow
db766bb073 [fix](planner) decimalv2 castTo decimalv2 should change type directly (#18297) 2023-04-06 13:51:50 +08:00
AKIRA
4ec6aa1691 [fix](planner) trying register constnat slotRef to table cause NPE (#18356) 
could reproduced by:

CREATE TABLE t (
name varchar(128)
) ENGINE=OLAP
UNIQUE KEY(name)
DISTRIBUTED BY HASH(name) BUCKETS 1;

insert into t values('abc');

SELECT cd
FROM
(SELECT cast(now() as string) cd FROM t) t1
JOIN
(select cast(now() as string) td from t GROUP BY now()) t2
ON t1.cd = t2.td;

ERROR 1105 (HY000): errCode = 2, detailMessage = Unexpected exception: null
 2023-04-06 11:50:12 +08:00
Jerry Hu
66a0c090b8 [fix](column) Add unimplemented replicate function in ColumnStruct (#18368) 2023-04-06 09:50:27 +08:00
ZhangYu0123
8b85c55117 [vectorized](function) Support array_shuffle and shuffle function. (#18116) 
---------

Co-authored-by: zhangyu209 <zhangyu209@meituan.com>
 2023-04-04 08:53:13 +08:00
starocean999
88c5e64c4a [fix](nereids) fix bug of SelectMaterializedIndexWithAggregate rule (#18265) 
1. create a project node to adjust the output column position when a mv is selected in olap scan node
2. pass SlotReference's column info when call Alias's toSlot() method
3. should compare plan's logical properties when compare two plans after rewrite
 2023-04-03 22:32:43 +08:00
Liqf
961f5d1bb7 [feature](function)Add St_Angle/St_Azimuth function (#18293) 
Add St_Angle/St_azimuth function：
St_Angle：
Enter three point, which represent two intersecting lines. Returns the angle between these lines. Point 2 and point 1 represent the first line and point 2 and point 3 represent the second line. The angle between these lines is in radians, in the range [0, 2pi). The angle is measured clockwise from the first line to the second line.

`

mysql> SELECT ST_Angle(ST_Point(1, 0),ST_Point(0, 0),ST_Point(0, 1));
+----------------------------------------------------------------------+
| st_angle(st_point(1.0, 0.0), st_point(0.0, 0.0), st_point(0.0, 1.0)) |
+----------------------------------------------------------------------+
| 4.71238898038469 |
+----------------------------------------------------------------------+
1 row in set (0.04 sec)
`

St_azimuth：
Enter two point, and returns the azimuth of the line segment formed by points 1 and 2. The azimuth is the angle in radians measured between the line from point 1 facing true North to the line segment from point 1 to point 2.
`

mysql> SELECT st_azimuth(ST_Point(0, 0),ST_Point(1, 0));
+----------------------------------------------------+
| st_azimuth(st_point(0.0, 0.0), st_point(1.0, 0.0)) |
+----------------------------------------------------+
| 1.5707963267948966 |
+----------------------------------------------------+
1 row in set (0.04 sec)
 2023-04-03 13:01:59 +08:00
zhangstar333
94e3472050 [bug](function) fix count equal function return incorrect value (#18200) 
fix count equal function return incorrect value
 2023-04-03 11:20:36 +08:00
TengJianPing
7cd8f7c9ba [fix](grouping) fix coredump of grouping function for outer join (#18292) 
Result of functions grouping and grouping_id is always not nullable, but outer join will convert the result column to nullable when necessary, which will cause mismatch of column type and column object when executing unctions grouping and grouping_id.
 2023-04-03 09:35:31 +08:00