Commit Graph

1040 Commits

Author SHA1 Message Date
Pxl
da2d0bba74 [Chore](case) move some mv case from feut to p0 (#18475)
move some mv case from feut to p0
2023-04-11 11:48:55 +08:00
e562017801 [feature](table-metadata) support altering the property "light_schema_change" for the tables which created before 1.2 (#17704) 2023-04-11 11:09:43 +08:00
101737023c [Bug](round) fix wrong scale for round-like function (#18507) 2023-04-11 09:36:59 +08:00
735cd15a3d [fix](nereids) PushdownAliasThroughJoin should handle same column with different alias in project list (#18470) 2023-04-10 11:50:37 +08:00
8d4a7325a9 [regression](join)Add subquery join case to regression test #17865 2023-04-09 18:15:37 +08:00
8a4a92f658 [Enchancement](compatible) show dateV2/datetimeV2 to date/datetime (#18358)
show dateV2/datetimeV2 to date/datetime

modify show create table
modify desc table
use desc table all to get real type from column ColumnType
2023-04-09 10:34:14 +08:00
0517616242 [vectorized](function) support array_repeat function to be compatible with hive syntax (#18028)
---------

Co-authored-by: zhangyu209 <zhangyu209@meituan.com>
2023-04-08 15:50:28 +08:00
0b8bc51b72 [fix](inverted index) Fix key column match query failed (#18436)
* [fix](inverted index) Fix key column match query failed

* [chore](regression case) add regression case

* [fix] fix regression case no order by
2023-04-08 15:45:08 +08:00
d881d71cd1 [Bug](cast) Fix bug for cast function between datetimev2 and string (#18442)
Fix bug for cast function between datetimev2 and string
2023-04-07 22:02:15 +08:00
30f2abe5d3 [FIX](Map)fix calculate map offset in olap convertor (#18295)
Fix be core when load bigger kv data in one row for map.
2023-04-07 17:04:08 +08:00
926c6056c7 [refactor](config) Delete the environment variable enable_vectorized_engine (#18166) 2023-04-07 14:23:16 +08:00
c32adba1cf [Refactor](Pipeline) Refactor pipeline code to improve coverage (#18376)
Refactor pipeline code to improve coverage
2023-04-07 13:09:44 +08:00
Pxl
e77da1519a [Enchancement](materialized-view) adjust desc table all display fields (#18357)
adjust desc table all display fields
2023-04-07 11:14:17 +08:00
Pxl
267b690dad [Bug](materialized-view) fix materialized-view query match not consider with order by elements (#18384)
fix materialized-view query match not consider with order by elements
2023-04-07 11:11:18 +08:00
505f25c580 [fix](planner)use base index if the where clause is a constant value (#18367)
sql : select bitmap_empty() from d_table where true;
should always use base index instead of any mv, because the conjuncts is constant (true) and use none of the column from any mv
2023-04-07 09:15:00 +08:00
4e1cdb9ce7 [fix](agg_sort)fix bug of agg sort group concat with order by(#18447) 2023-04-07 08:42:36 +08:00
759f1da32e [Enhencement](Backends) add HostName filed in backends table and delete backends table in information_schema (#18156)
1.  Add `HostName` field for `show backends` statement and `backends()` tvf.
2. delete the `backends` table in `information_schema` database
2023-04-07 08:30:42 +08:00
550c8aa648 [Bug](DECIMALV3) fix wrong decimal scale returned by function round (#18375) 2023-04-06 14:44:21 +08:00
db766bb073 [fix](planner) decimalv2 castTo decimalv2 should change type directly (#18297) 2023-04-06 13:51:50 +08:00
8b61709ec8 [feature](multi-catalog) support select current_catalog(); (#18163) 2023-04-06 12:06:10 +08:00
4ec6aa1691 [fix](planner) trying register constnat slotRef to table cause NPE (#18356)
could reproduced by:

CREATE TABLE t (
name varchar(128)
) ENGINE=OLAP
UNIQUE KEY(name)
DISTRIBUTED BY HASH(name) BUCKETS 1;

insert into t values('abc');

SELECT cd
FROM
(SELECT cast(now() as string) cd FROM t) t1
JOIN
(select cast(now() as string) td from t GROUP BY now()) t2
ON t1.cd = t2.td;

ERROR 1105 (HY000): errCode = 2, detailMessage = Unexpected exception: null
2023-04-06 11:50:12 +08:00
4ca0c0face [fix](join) fix wrong result of right join (#18365)
When processing data in hash table for right join and full outer join, if the output data rows of one hash bucket excceeds batch size, the logic when continue processing this bucket is wrong, it should differentiate between different join types.
2023-04-06 10:55:58 +08:00
66a0c090b8 [fix](column) Add unimplemented replicate function in ColumnStruct (#18368) 2023-04-06 09:50:27 +08:00
7c36bef6bc [Feature-Wip](MySQL Load)Show load warning for my sql load (#18224)
1. Support the show load warnings for mysql load to get the detail error message.
2. Fix fillByteBufferAsync not mark the load as finished in same data load
3. Fix drain data only in client mode.
2023-04-04 22:44:48 +08:00
87e83081ff [test](compaction) add delete test (#18335) 2023-04-04 12:28:19 +08:00
54dbb4af67 [vectorzied](jdbc) refactor jdbc table read array type (#18187)
jdbc read array type get result from Doris is string, PG is java.sql.array, CK is java.lang.object
it's difficult to maintain and read the code,
so change all database's array result to string, then add a cast function from string to doris array type
2023-04-04 11:57:04 +08:00
8b85c55117 [vectorized](function) Support array_shuffle and shuffle function. (#18116)
---------

Co-authored-by: zhangyu209 <zhangyu209@meituan.com>
2023-04-04 08:53:13 +08:00
eb0fd0017e [Fix](orc-reader) Fix the scale of decimal column is incorrect when query orc tables. (#18324)
The scale of decimal column is incorrect when query orc tables.
2023-04-04 08:50:47 +08:00
88c5e64c4a [fix](nereids) fix bug of SelectMaterializedIndexWithAggregate rule (#18265)
1. create a project node to adjust the output column position when a mv is selected in olap scan node
2. pass SlotReference's column info when call Alias's toSlot() method
3. should compare plan's logical properties when compare two plans after rewrite
2023-04-03 22:32:43 +08:00
fe9d2b00fc [test](jdbc catalog) add clickhouse jdbc catalog base type test (#18007) 2023-04-03 20:18:36 +08:00
3078ee1854 [regression](decimalv3)Add decimal type as filter condition in regression test (#17160)
Add decimal type as filter condition in regression test
2023-04-03 14:20:09 +08:00
ecd3fd07f6 [feature](colocate) support cross database colocate join (#18152) 2023-04-03 14:03:42 +08:00
961f5d1bb7 [feature](function)Add St_Angle/St_Azimuth function (#18293)
Add St_Angle/St_azimuth function:
St_Angle:
Enter three point, which represent two intersecting lines. Returns the angle between these lines. Point 2 and point 1 represent the first line and point 2 and point 3 represent the second line. The angle between these lines is in radians, in the range [0, 2pi). The angle is measured clockwise from the first line to the second line.

`

mysql> SELECT ST_Angle(ST_Point(1, 0),ST_Point(0, 0),ST_Point(0, 1));
+----------------------------------------------------------------------+
| st_angle(st_point(1.0, 0.0), st_point(0.0, 0.0), st_point(0.0, 1.0)) |
+----------------------------------------------------------------------+
| 4.71238898038469 |
+----------------------------------------------------------------------+
1 row in set (0.04 sec)
`

St_azimuth:
Enter two point, and returns the azimuth of the line segment formed by points 1 and 2. The azimuth is the angle in radians measured between the line from point 1 facing true North to the line segment from point 1 to point 2.
`

mysql> SELECT st_azimuth(ST_Point(0, 0),ST_Point(1, 0));
+----------------------------------------------------+
| st_azimuth(st_point(0.0, 0.0), st_point(1.0, 0.0)) |
+----------------------------------------------------+
| 1.5707963267948966 |
+----------------------------------------------------+
1 row in set (0.04 sec)
2023-04-03 13:01:59 +08:00
Pxl
e77833bfa1 [Bug](materialized-view) fix where clause persistence replay incorrect (#18228)
fix where clause persistence replay incorrect
2023-04-03 12:49:01 +08:00
94e3472050 [bug](function) fix count equal function return incorrect value (#18200)
fix count equal function return incorrect value
2023-04-03 11:20:36 +08:00
2bce4db81a [Enchancement](mysql-compatable) add regression-test for MySQLdump #18208
add regression-test for like this:
mysqldump -h127.0.0.1 -P9030 -uroot --no-tablespaces --databases > /backup/mysqldump/test.db

To prevent errors Unknown table 'column_statistics' in information_schema (1109), the table information_schema.column_statistics was added.
2023-04-03 09:49:07 +08:00
7cd8f7c9ba [fix](grouping) fix coredump of grouping function for outer join (#18292)
Result of functions grouping and grouping_id is always not nullable, but outer join will convert the result column to nullable when necessary, which will cause mismatch of column type and column object when executing unctions grouping and grouping_id.
2023-04-03 09:35:31 +08:00
97aab138aa [fix](parquet-reader) reset value idx in bool rle decoder and support iceberg datetime(3) (#18245)
1. Fix value  idx in bool rle decoder 
2. Iceberg table support datetimev2(3).  In the previous version, we converted hive timestamp to datetimev2(0) default.
2023-04-01 21:00:01 +08:00
Pxl
236ee1411e [Chore](case) add p0 testSubQuery,testJoinOnLeftProjectToJoin,testAggregateMVCalcAg 2023-03-31 19:46:05 +08:00
20b3bdb000 [vectorized](function) support array_first_index function (#18175)
mysql> select array_first_index(x->x+1>3, [2, 3, 4]);
+-------------------------------------------------------------------+
| array_first_index(array_map([x] -> x(0) + 1 > 3, ARRAY(2, 3, 4))) |
+-------------------------------------------------------------------+
|                                                                 2 |
+-------------------------------------------------------------------+

mysql> select array_first_index(x -> x is null, [null, 1, 2]);
+----------------------------------------------------------------------+
| array_first_index(array_map([x] -> x(0) IS NULL, ARRAY(NULL, 1, 2))) |
+----------------------------------------------------------------------+
|                                                                    1 |
+----------------------------------------------------------------------+

mysql> select array_first_index(x->power(x,2)>10, [1, 2, 3, 4]);
+---------------------------------------------------------------------------------+
| array_first_index(array_map([x] -> power(x(0), 2.0) > 10.0, ARRAY(1, 2, 3, 4))) |
+---------------------------------------------------------------------------------+
|                                                                               4 |
+---------------------------------------------------------------------------------+
2023-03-31 12:51:29 +08:00
Pxl
307170030c [Bug](materialized-view) fix core dump when create mv have case different with base table (#18206)
fix core dump when create mv have case different with base table
2023-03-31 12:32:09 +08:00
1b2aaab2f2 [vectorized](bug) fix some case in enable fold constant (#17997)
fix some case in enable fold constant
2023-03-31 11:41:31 +08:00
Pxl
e7bcd970f5 [Bug](materialized-view) fix isDisableTuplesMVRewriter rreturn true when expr is literal (#18246)
fix isDisableTuplesMVRewriter rreturn true when expr is literal
2023-03-31 11:30:47 +08:00
1938899aa3 [regression-test] add grouping sets test case (#18194) 2023-03-31 11:00:38 +08:00
4e1e0ce06d [bugfix](topn) fix topn optimzation wrong result for NULL values (#18121)
1. add PassNullPredicate to fix topn wrong result for NULL values
2. refactor RuntimePredicate to avoid using TCondition
3. refactor using ordering_exprs in fe and vsort_node
2023-03-31 10:01:34 +08:00
b5ea299697 [fix](planner) Fix agg on inlineview which with constant slot (#18201)
Since slot that reference to constant has been marked as constant expr either, just add condition check to make sure such slot wouldn't be eliminated as constant from group exprs
2023-03-30 23:54:37 +08:00
99bd5ec022 [fix](Nereids) fix some bugs in Subquery to window rule (#18233)
we introduce this rule by PR #17968, but some corner case do
not be processed correctly. This PR fix these bugs:
1. fix window function generation method, replace inner slot with
   equivalent outer slot
2. forbid below scenes
    a. inner has a mapping project
    b. inner has an unexpected filter
    c. outer has a mapping project
    d. outer has an unexpected filter
    e. outer has additional table
    f. outer has same table
    g. outer and inner with different join condition
    h. outer and inner has same table with different join condition
2023-03-30 16:09:16 +08:00
ea41d94582 [Improve](complex-type) Support Count(complexType) (#17868)
Support count function for ARRAY/MAP/STRUCT type
2023-03-30 15:43:32 +08:00
e3bd812887 [fix](stream-load) find line delimiter in csv should start with no offset (#18161)
when loading big file with multi bytes line delimiter, some line record maybe incomplete because of _output_buf_limit, so this incomplete data will move to the beginning of the output buf and read more data into output buf. In this case, find line delimiter should start with no offset to avoid a bug that spilt two lines as one line.
2023-03-30 14:42:34 +08:00
Pxl
c8ad62a3cd [Enchancement](materialized-view) enchance materialized view where clause match (#18179)
enchance materialized view where clause match
2023-03-30 13:02:21 +08:00