Only write editlog for manual analyze task, don't need to do so for auto tasks to reduce writing editlog.
Add error message to job info while task failed.
Support to create partition materialized view using nodata table
Such as the table def as following:
> CREATE TABLE `test_no_data` (
> `user_id` LARGEINT NOT NULL COMMENT '"用户id"',
> `date` DATE NOT NULL COMMENT '"数据灌入日期时间"',
> `num` SMALLINT NOT NULL COMMENT '"数量"'
> ) ENGINE=OLAP
> DUPLICATE KEY(`user_id`, `date`, `num`)
> COMMENT 'OLAP'
> PARTITION BY RANGE(`date`)
> (PARTITION p201701_1000 VALUES [('0000-01-01'), ('2017-02-01')),
> PARTITION p201702_2000 VALUES [('2017-02-01'), ('2017-03-01')),
> PARTITION p201703_all VALUES [('2017-03-01'), ('2017-04-01')))
> DISTRIBUTED BY HASH(`user_id`) BUCKETS 2
> PROPERTIES ('replication_num' = '1') ;
when table test_no_data has no data, it also support to create partition materialized view as following:
> CREATE MATERIALIZED VIEW no_data_partition_mv
> BUILD IMMEDIATE REFRESH AUTO ON MANUAL
> partition by(`date`)
> DISTRIBUTED BY RANDOM BUCKETS 2
> PROPERTIES ('replication_num' = '1')
> AS
> SELECT * FROM test_no_data where date > '2017-05-01';
>
Query rewrite by mv support bitmap_union and bitmap_union_count roll up, aggregate functions which supports roll up is listed as following:
| 查询中函数 | 物化视图中函数 | 函数上卷后 |
|------------------|--------------|--------------------|
| max | max | max |
| min | min | min |
| sum | sum | sum |
| count | count | sum |
| count(distinct ) | bitmap_union | bitmap_union_count |
| bitmap_union | bitmap_union | bitmap_union|
| bitmap_union_count | bitmap_union | bitmap_union_count |
this depends on https://github.com/apache/doris/pull/29256
The current logic for SQL dialect conversion is all in the `fe-core` module, which may lead to the following issues:
- Changes to the dialect conversion logic may occur frequently, requiring users to upgrade the Doris version frequently within the fe-core module, leading to a longer change cycle.
- The cost of customized development is high, requiring users to replace the fe-core JAR package.
Turning it into a plugin can address the above issues properly.
Problem:
fe ut failed cause of null pointer error
Cause:
fe ut getting statement context from connection context failed
Resolved:
add null pointer judgement
Force to use zonemap for collecting string type min max.
String type is not using zonemap for min max, because zonemap value at BE side is truncated at 512 bytes which may cause the value not accurate. But it's OK for statisitcs min max, and this could also avoid scan whole table while sampling.
Be do not support RF for NullSafeEquals, so fe not generate RF for them.
However, after we support NullSafeEquals as Hash join condition,
the order of RF is wrong when generating RF in FE. this PR fix it.
The signatures of functions in these PRs should be more standard:
#27342,
#25510,
#20936,
including the following:
ipv4numtostring,
ipv4stringtonum,
ipv4stringtonumordefault,
ipv4stringtonumornull,
ipv6numtostring.
This PR will add necessary underscores between the words of each of them,
like changing ipv4numtostring to ipv4_num_to_string.
Modify computeSampleTabletIds location. table sample not take effect because selectedIndexId=-1 in computeSampleTabletIds, the previous run FE UT had special processing, so FE UT did not discover the BUG in time.
