should not push down topn distinct through join when the output
columns of the corresponding child of join is more than
aggregate distinct columns.
for example for LEFT_OUTER_JOIN:
left child of join's output is: c1, c2, c3.
distinct columns is: c1, c2
topn: limit 2
if we push down topn distinct, we could get result of join like this:
```
c1 c2 c3, ...
1 2 1
1 2 2
```
and the final result we get is:
```
c1 c2
1 2
```
this is wrong, because we need 2 lines, but only return 1.
this PR support slot appearing both in agg func and grouping sets.
sql like below:
select sum(a) from t group by grouping sets ((a));
Before this PR, Nereids throw exception like below:
col_int_undef_signed cannot both in select list and aggregate functions when using GROUPING SETS/CUBE/ROLLUP, please use union instead.
This PR removes the restriction and supports this situation.
Issue Number: close #xxx
For example, the hive table is partitioned by `date` and `region`, with the following 6 partitions
```
20200101
beijing
shanghai
20200102
beijing
shanghai
20200103
beijing
shanghai
```
If the MTMV is partitioned by `date`, then the MTMV will have three partitions: 20200101, 202000102, 20200103
If the MTMV is partitioned by `region`, then the MTMV will have two partitions: beijing, shanghai
1. run PushDownFilterThroughAggregation, PushDownFilterThroughProject and MergeFilters before subquery unnesting
2. should keep plan unchanged even left semi join's condition is true (because the right table may be empty())
3. PushDownFilterThroughProject need check if filter's input slots are all coming from project's output
* (feature)(cloud) Use dynamic allocator instead of static buffer pool for better elasticity. (#28983)
* [fix](outfile) Fix unable to export empty data (#30703)
Issue Number: close#30600
Fix unable to export empty data to hdfs / S3, this behavior is inconsistent with version 1.2.7,
version 1.2.7 can export empty data to hdfs/ S3, and there will be exported files on S3/HDFS.
* [fix](file-writer) avoid empty file for segment writer (#31169)
---------
Co-authored-by: AlexYue <yj976240184@gmail.com>
Co-authored-by: zxealous <zhouchangyue@baidu.com>
In the previous design, we were compatible with MySQL's auto-increment column and default value to bypass the null value check when writing back Jdbc External Table. However, because MySQL's default value is not completely unified with Doris, this resulted in The unsuitable default value is wrong. In response to this situation, I made the following optimizations
1. For JDBC External Table, we always allow certain columns to be missing during insertion. Even if these columns are not allowed to be empty at the source end, the error should be generated by the source end, not Doris herself.
2. When the target column is non-nullable and the insertion is done via `INSERT INTO tbl VALUES()` or `INSERT INTO tbl SELECT constants`, Doris should verify any inconsistency between them and throw an exception. This check is not applied for `INSERT INTO tbl SELECT ... FROM tbl` operations.
