Commit Graph

114 Commits

Author SHA1 Message Date
35c0697a60 [fix](planner)SelectStmt's constructor should initialize originSelectList member (#24755)
resetSelectList method will use originSelectList to recover the origin select list. If the originSelectList is lost in constructor, the resetSelectList will fail to recover and make the analyze process fail.
2023-09-22 19:31:29 +08:00
c5e7f55b63 [performance](executor) optimize time_round function (#23058)
optimize time_round function
2023-09-15 10:49:22 +08:00
ccba5a729a [fix](planner)cast string to float like type should return NULL literal if it fails (#24222) 2023-09-14 15:59:20 +08:00
21e30d4374 [fix](planner)ctas's query part is not analyzed correctly (#24111)
* [fix](planner)ctas's query part is not analyzed correctly
2023-09-09 20:55:09 +08:00
37b90021b7 [fix](planner)literal expr should do nothing in substituteImpl() method (#23438)
substitute a literal expr is pointless and wrong. This pr keep literal expr unchanged during substitute process
2023-08-25 11:21:35 +08:00
441a9fff6d [fix](planner) fix now function param type error (#23446) 2023-08-25 00:12:21 +08:00
1b7d692d72 [fix](planner & nereids) convert to double if div decimal overflow (#23272) 2023-08-23 20:10:53 +08:00
78c6b115c3 [fix](planner)avg function need support large int param (#23254)
* [fix](planner)avg function need support large int param
2023-08-23 10:05:08 +08:00
a7771ea507 [fix](planner) fix current_timestamp param type mismatch when doing stream load (#23092)
FileLoadScanNode did not analyze the default value expr, result in target param type int32 become int8 as the original IntLiteral type.
2023-08-18 14:28:45 +08:00
e289e03a1a [fix](executor)fix no return with old type in time_round 2023-08-17 15:34:26 +08:00
548226acfc [fix](planner)shouldn't change the child type to assignmentCompatibleType if it's INVALID_TYPE (#22841)
if changing the child type to INVALID_TYPE, the later getBuiltinFunction call will fail
2023-08-11 17:14:49 +08:00
1617368ee1 [fix](planner) fix bug of push constant conjuncts through set operation node (#22695)
when pushing down constant conjunct into set operation node, we should assign the conjunct to agg node if there is one. This is consistant with pushing constant conjunct into inlineview.
2023-08-08 12:25:42 +08:00
c31226b144 [refractor](regression-test) sort out test cases of external tables (#22640)
sort out the test cases of external table.
After modify, there are 2 directories:

1. `external_table_p0`: all p0 cases of external tables: hive, es, jdbc and tvf
2. `external_table_p2`: all p2 cases of external tables: hive, es, mysql, pg, iceberg and tvf

So that we can run it with one line command like:

```
sh run-regression-test.sh --run -d external_table_p0,external_table_p2
```
2023-08-07 11:12:30 +08:00
ca6e33ec0c [feature](table-value-functions)add catalogs table-value-function (#21790)
mysql> select * from catalogs() order by CatalogId;
2023-07-14 10:25:16 +08:00
35fa9496e7 [fix](merge-on-write) fix wrong result when query with prefix key predicate (#21770) 2023-07-13 19:56:00 +08:00
d3be10ee58 [improvement](column) Support for the default value of current_timestamp in microsecond (#21487) 2023-07-11 14:04:13 +08:00
2e651bbc9a [fix](nereids) fix some planner bugs (#21533)
1. allow cast boolean as date like type in nereids, the result is null
2. PruneOlapScanTablet rule can prune tablet even if a mv index is selected.
3. constant conjunct should not be pushed through agg node in old planner
2023-07-06 16:13:37 +08:00
599ba4529c [fix](nereids) need run ConvertInnerOrCrossJoin rule again after EliminateNotNull (#21346)
after running EliminateNotNull rule, the join conjuncts may be removed from inner join node.
So need run ConvertInnerOrCrossJoin rule to convert inner join with no join conjuncts to cross join node.
2023-07-04 10:52:36 +08:00
b1e973b721 [Improve](func)support array to window-func first-last-value arg type (#21201)
* support array to windown-func first-last-value arg type

* add regress test for first-last-value of array type

* update

* format be:
2023-06-28 10:02:00 +08:00
84554ec0fd [fix](planner) the resultExprs should be substituted using table function node's outputSmap (#21182) 2023-06-27 17:19:49 +08:00
Pxl
a0d4f11667 [Bug](function) catch error state in function cast to avoid core dump (#20751)
catch error state in function cast to avoid core dump
2023-06-14 17:34:34 +08:00
7942bd0bf9 [fix](planner) cast string literal to date like type should not be an implict cast (#20709)
1. cast string literal to date like type should not be an implict cast
2. the string representation of float like type should not be scientific notation
3. the data type of like function's regex expr should be string type even if it's a null literal
4. add -Xss4m in fe.conf to prevent stack overflow in some case
2023-06-13 17:57:14 +08:00
0b228b3414 [fix](load)Support load json data with default value (#20624)
* support json default value

---------

Co-authored-by: duanxujian <duanxujian@jd.com>
2023-06-12 14:51:31 +08:00
bcc37c9405 [fix](planner)the common type of floating and decimal should be floating type (#20634)
* [fix](planner)the common type of floating and decimal should be floating type

* fix test cases
2023-06-12 11:32:23 +08:00
5b2efd196b [fix](execution) result_filter_data should be filled by 0 when can_filter_all is true (#20438) 2023-06-05 17:05:35 +08:00
519f01133a [feature](decimal)support cast rounding half up and div precision increment in decimalv3. (#19811) 2023-06-01 13:09:58 +08:00
4cbb6ece10 [fix](fe)ordering exprs should be substituted in the same way as select part (#20091) 2023-05-27 21:00:57 +08:00
558f625d3b [fix](planner) The group by part should be substituted in the same way as select part (#20019) 2023-05-26 11:05:02 +08:00
f43e8cc98f [regressiontest](unionall) Regression_test_similar_query_boolean (#19553)
* regression_test_similar_query

* add the ORDER BY

* update ORDER BY to comfirm correctness

---------

Co-authored-by: ZI-MA <chime316@qq.com>
2023-05-18 12:21:32 +08:00
88ca4f3e6b [feature](like) make like regexp used as a sql function (#19755) 2023-05-18 10:03:12 +08:00
9535ed01aa [feature](tvf) Support compress file for tvf hdfs() and s3() (#19530)
We can support this by add a new properties for tvf, like :

`select * from hdfs("uri" = "xxx", ..., "compress_type" = "lz4", ...)`

User can:

Specify compression explicitly by setting `"compression" = "xxx"`.
Doris can infer the compression type by the suffix of file name(e.g. `file1.gz`)
Currently, we only support reading compress file in `csv` format, and on BE side, we already support.
All need to do is to analyze the `"compress_type"` on FE side and pass it to BE.
2023-05-16 08:50:43 +08:00
92bf485abd [Bug] Fix doris pipeline shared scan and top n opt (#19599) 2023-05-15 10:00:44 +08:00
af04c3acab [fix](sequence-column) Fix sequence_col column used default expr insert failed (#18933) 2023-05-08 17:18:25 +08:00
3e3262361c [fix](fe)havingClause should be substituted the same way as resultExprs (#19261)
substituted havingClause in the same way as resultExprs to prevent " HAVING clause not produced by aggregation output" error
2023-05-05 18:03:43 +08:00
aacc075f09 [fix](planner) SetOperationNode's slots' nullability calculation is wrong (#19108)
SetOperationNode's slots' nullability should consider slots info from all children, even some children have EmptyResultSet
2023-04-26 21:18:37 +08:00
61b7a52444 [Enhancement](multi-catalogs) Use decimal V3 type in multi-catalogs module. (#18926)
1. Use decimal V3 type in JDBC and Iceberg tables.
2. Fix hdfs TVF decimal V3 type and regression test.
2023-04-25 14:49:40 +08:00
3328a65b75 [Fix](mutli-catalog) Use decimal v3 type to fix decimal loss issue in multi-catalog module. (#18835)
Fix decimal v3 precision loss issues in the multi-catalog module.
Now it will use decimal v3 to represent decimal type in the multi-catalog module.
Regression Test: `test_load_with_decimal.groovy`
2023-04-20 11:02:53 +08:00
aa6b3cc537 [fix](planner)keep all agg functions if there is any virtual slots in group by list (#18630)
Because of the limitation of ProjectPlanner, we have to keep set agg functions materialized if there is any virtual slots in the group by list, such as 'GROUPING_ID' in the group by list etc.
2023-04-13 19:44:46 +08:00
43392918cd [Optimization](functions)Optimize function call for const columns. (#18310) 2023-04-12 11:11:01 +08:00
ecd3fd07f6 [feature](colocate) support cross database colocate join (#18152) 2023-04-03 14:03:42 +08:00
Pxl
e77833bfa1 [Bug](materialized-view) fix where clause persistence replay incorrect (#18228)
fix where clause persistence replay incorrect
2023-04-03 12:49:01 +08:00
d27201f331 [fix](nested_loop_join)got incorrect result from nested loop join without condition (#18139) 2023-03-28 16:20:05 +08:00
d7dcdfcba9 [Fix](Create View) support create view from tvf (#18087)
Support create view as select * from tvf()
2023-03-28 15:07:32 +08:00
8ffc85b6ff [fix](planner)project should be done inside inlineview (#17831)
* [fix](planner)project should be done inside inlineview

* add src column for slots in scan node's output tuple
2023-03-20 21:12:45 +08:00
782001c75b [fix](planner) project should be done inside subquery (#17630)
WITH t0 AS(
SELECT report.date1 AS date2 FROM(
SELECT DATE_FORMAT(date, '%Y%m%d') AS date1 FROM cir_1756_t1
) report GROUP BY report.date1
),
t3 AS(
SELECT date_format(date, '%Y%m%d') AS date3
FROM cir_1756_t2
)
SELECT row_number() OVER(ORDER BY date2)
FROM(
SELECT t0.date2 FROM t0 LEFT JOIN t3 ON t0.date2 = t3.date3
) tx;

The DATE_FORMAT(date, '%Y%m%d') was calculated in GROUP BY node, which is wrong. This expr should be calculated inside the subquery.
2023-03-13 11:10:27 +08:00
13e05c4a5d [Enhencement](stream load) add some regression test for json format streamload (#17520) 2023-03-12 20:13:07 +08:00
08f0170895 [fix](olap) The 'scan key' generated by the 'is null' expression causes incorrect query results (#17569) 2023-03-10 08:51:06 +08:00
caacee253d [fix](olap)Crashing caused by IS NULL expression (#17463)
Issue Number: close #17462
2023-03-07 15:32:52 +08:00
479d57df88 [fix](planner) the project expr should be calculated in join node in some case (#17035)
Consider the sql bellow:

select sum(cc.qlnm) as qlnm
FROM
  outerjoin_A
  left join (SELECT
      outerjoin_B.b,
      coalesce(outerjoin_C.c, 0) AS qlnm
    FROM
      outerjoin_B
      inner JOIN outerjoin_C ON outerjoin_B.b = outerjoin_C.c
  ) cc on outerjoin_A.a = cc.b
group by outerjoin_A.a;

The coalesce(outerjoin_C.c, 0) was calculated in the agg node, which is wrong.
This pr correct this, and the expr is calculated in the inner join node now.
2023-02-24 15:20:05 +08:00
90af1b0113 [fix](subquery) fix bug of using constexpr and some agg func(like count,max) as subquery's output (#16579)
Signed-off-by: nextdreamblue <zxw520blue1@163.com>
2023-02-14 00:11:56 +08:00