This PR was originally #16940 , but it has not been updated for a long time due to the original author @Cai-Yao . At present, we will merge some of the code into the master first.
thanks @Cai-Yao @yiguolei
when pushing down constant conjunct into set operation node, we should assign the conjunct to agg node if there is one. This is consistant with pushing constant conjunct into inlineview.
count_by_enum(expr1, expr2, ... , exprN);
Treats the data in a column as an enumeration and counts the number of values in each enumeration. Returns the number of enumerated values for each column, and the number of non-null values versus the number of null values.
Problem:
When window function in aggregation function, executor would report an error like: Required field 'node_type' was not present!
Example:
SELECT SUM(MAX(c1) OVER (PARTITION BY c2, c3)) FROM test_window_in_agg;
Reason:
When analyze aggregate, analytic expr (window function carrior when analyze) transfered to slot and loss message. So when
serialize to thrift package, TExpr can not determine node_type of analytic expr.
Solved:
We do not support aggregate(window function) yet. So we report an error when analyze.
* Revert "[fix](executor) only mysql connect to set GlobalPipelineTask (#22205)"
* Revert "[feature](executor) using fe version to set instance_num (#22047)"
1. Remove bunch of SQLs related to partition's information
2. Fix the duplicate SQLs submission
3. Fix bug that table's stats not get updated after system job finished
Hive file path may contain temporary directory like this:
drwxrwxrwx - root supergroup 0 2023-03-22 21:03 /usr/hive/warehouse/datalake_performance.db/clickbench_parquet_hits/.hive-staging_hive_2023-03-22_21-03-12_047_8461238469577574033-1
drwxrwxrwx - root supergroup 0 2023-05-18 15:03 /usr/hive/warehouse/datalake_performance.db/clickbench_parquet_hits/.hive-staging_hive_2023-05-18_15-03-52_780_3065787006787646235-1
This will cause error when be try to read these files. Need to filter them during FE plan.
The error message was not good for not exist column while show column stats:
```
MySQL [hive.tpch100]> show column stats `lineitem` (l_extendedpric);
ERROR 1105 (HY000): errCode = 2, detailMessage = Unexpected exception: null
```
This pr show a meaningful message:
```
mysql> show column stats `lineitem` (l_extendedpric);
ERROR 1105 (HY000): errCode = 2, detailMessage = Column: l_extendedpric not exists
```
use three new plan node to represent defer materialize of TopN.
Example:
```
-- SQL
select * from t1 order by c1 limit 10;
-- PLAN
+------------------------------------------+
| Explain String |
+------------------------------------------+
| PhysicalDeferMaterializeResultSink |
| --PhysicalDeferMaterializeTopN |
| ----PhysicalDistribute |
| ------PhysicalDeferMaterializeTopN |
| --------PhysicalDeferMaterializeOlapScan |
+------------------------------------------+
```
