6502da8917
[bugfix](restore) add partition id into convert_rowset_ids() ( #24834 )
2023-09-25 20:07:24 +08:00
9dff620471
[feature](binlog) add truncate table binlog ( #24868 )
...
Signed-off-by: Jack Drogon <jack.xsuperman@gmail.com >
2023-09-25 19:53:42 +08:00
3e686f4306
[feature](javaudf)support no input java udf ( #24457 )
...
* support no input java udf
* add license
2023-09-25 18:56:44 +08:00
8679095e5c
[feature](debug) support debug point used in debug code ( #24502 )
2023-09-25 17:56:12 +08:00
90791f0b19
[fix](nereids)fix bug of exists subquery with limit clause ( #24630 )
...
create table t1(c1 int, c2 int);
create table t2(c1 int, c2 int);
insert into t1 values (1,1);
insert into t2 values (1,1);
select * from t1 where exists (select * from t2 where t1.c1 = t2.c1 limit 0);
the result should be empty set.
2023-09-25 17:15:08 +08:00
60af62b41e
[enhancement](Nereids) remove deriveStats jobs for some groupExpression ( #24835 )
...
Don't call DeriveStatJob for follow group expressions:
- the group expression that is generated by the joinCommute rule
- the group expression that is generated by the implementation rule without creating a new group
2023-09-25 17:14:16 +08:00
1b95ce1d93
[feature](json-function) add json_insert, json_replace, json_set functions ( #24384 )
...
[feature](json-function) add three json funcitons
2023-09-25 12:52:29 +08:00
82b393314c
[enhancement](Nereids) check unimplemented func in nereids ( #24766 )
2023-09-25 12:45:11 +08:00
2d0ac7117a
[enhancement](show-frontends-disks) Add disk info corresponding to deploy dir in frontend disk info ( #24726 )
2023-09-25 11:14:03 +08:00
ec93ea22b3
[bugfix](Coordinator) The error msg hide host info exclude localhost ( #24817 )
...
Exception information is incomplete when localhost exists in the log.
2023-09-25 09:45:00 +08:00
8fd27a852c
[fix](transaction) fix profile null exception cause txn stuck ( #24813 )
2023-09-24 23:46:38 +08:00
16aee69911
[improvement](tablet schedule) disk balance skip cooldown storage ( #24580 )
2023-09-24 23:09:29 +08:00
900b06d14e
[improvement](disk balancer) reduce disk balance sensitivity ( #24637 )
2023-09-24 22:02:53 +08:00
ece2e3cd70
[fix](nereids) support cte rf pushdown for exttable ( #24806 )
...
Currently, cte inside rf pushdown doesn't support external table, this pr open this restriction.
And since rf for schema scan, for example tables under information_schema, currently isn't supported by be, this pr also check and prevent to generate rf on them.
2023-09-23 23:00:05 +08:00
ce8dfd3561
[Chore](dependency)grpc library dependencies are unified ( #24794 )
2023-09-23 14:29:34 +08:00
bf07415ff0
[fix](es catalog) remove _default_ field when parsing mapping. ( #24781 )
2023-09-23 10:58:55 +08:00
8ca300f70b
[Opt](pipeline) disable shared scan in key point search ( #24797 )
2023-09-22 22:46:11 +08:00
b86f09418f
[chore](build) Fix the FE build on CentOS 6 ( #24798 )
...
Using grpc-java whose version is newer than 1.34.0 will break the build on CentOS 6 due to the obsolete GLIBC.
2023-09-22 19:58:12 +08:00
35c0697a60
[fix](planner)SelectStmt's constructor should initialize originSelectList member ( #24755 )
...
resetSelectList method will use originSelectList to recover the origin select list. If the originSelectList is lost in constructor, the resetSelectList will fail to recover and make the analyze process fail.
2023-09-22 19:31:29 +08:00
eb9d334b55
[opt](nereids) enable runtime filter prune by default ( #24717 )
...
add stats back to physical plan in post processors. stats are used by rf-prune
set enable_runtime_filter_prune true by default
2023-09-22 19:11:10 +08:00
c943a05065
[fix](stats) Fix data size calculation of auto sample ( #24672 )
...
1. Fix data size calculation of auto sample, before this pr, the data size is include all the replicas
2. Move some auto analyze related options to global session variable
3. Add some logs
2023-09-22 18:12:39 +08:00
b34a4779e5
[feat](optimizer) calculate stats health based on real updated rows count ( #24599 )
...
Use actual load rows since last analyze rather than delta of total row count.
2023-09-22 18:12:13 +08:00
85a1fbd5d3
[Improve](stats)Use Log4j class library instead of Quartz ( #24732 )
...
Quartz new version not support java 8
2023-09-22 15:23:58 +08:00
4edba083c0
[feat](Nereids) Support tablesample syntax ( #23717 )
...
Add table sample support like such SQL:
```sql
select * from test_table_sample_tbl tablesample(4 rows);
select * from test_table_sample_tbl t tablesample(20 percent);
select * from test_table_sample_tbl t tablesample(20 percent) repeatable 2;
```
This function has already been implemented in legacy planner, more detailed description could be found here: https://github.com/apache/doris/pull/10170
2023-09-22 14:24:07 +08:00
320fc1481a
[fix](Nereids) some expression not cast in in predicate ( #24680 )
...
1. should use castIfNotSameType in InPredicate and CaseWhen
2. StringLikeLiteral should override equals to ignore type
2023-09-22 12:58:33 +08:00
22616d125d
[function](bitmap) add function alias bitmap_andnot and bitmap_andnot_count ( #24771 )
2023-09-22 12:18:31 +08:00
b5f6ace204
[fix](planner) ctas should not change any meta of column in source table ( #24767 )
...
if previous PR #22770 . we try to fix incorrect nullable in target table.
However we changed nullable info of column in source table unexpectly
2023-09-22 11:49:38 +08:00
79fe9d58b0
[bug](expr) forget write date_liter expr code info ( #24687 )
...
[bug](expr) forget write date_liter expr code info
2023-09-22 11:08:41 +08:00
a718f1f6cd
[fix](lock): do not use instance lock to protect static data ( #24611 )
2023-09-22 10:16:54 +08:00
c832e018d0
[Dependence](Fe)Upgrade Fe dependencies ( #24606 )
...
* be scanner
- Upgrade avro to 1.11.2
fe
- Upgrade quartz to 2.5.0-rc1
- Upgrade maxcompute to 0.45-2-publish
- Binding avro-ipc to 1.11.2
* Binding hbase version to 2.5.5
binding nimbusds version to 9.35
2023-09-22 10:14:42 +08:00
090be20ca4
[cases](regresstests) add negative case for agg table and fix agg table support replace typ… #24715
...
add negative case for agg table
fix agg table support replace agg type for complex type , and Now We only support complex type with agg state for replace only
fix test output
2023-09-22 09:05:20 +08:00
e4c55cc5aa
[fix](web): field in Controller is singleton, which would introduce thread safe problem, so remove it ( #24667 )
...
Co-authored-by: yiguolei <676222867@qq.com >
2023-09-22 08:58:01 +08:00
6685875e72
[Improvement](statistics)Mark unfinished analysis job failed after master reboot
...
Before, the Pending/Running analysis tasks/jobs will stay in the unfinished status for ever after FE reboot, which is misleading.
In this pr, jobs/tasks are only logged to editlog when they finish. So the unfinished tasks/jobs are abandoned after reboot.
Also return without retry when analyze table cancelled by user in with sync mode.
2023-09-21 23:11:50 +08:00
48f3614ce8
[fix](nereids) a bug in estimation of intersect ndv ( #24664 )
2023-09-21 23:06:31 +08:00
a17034b7ff
[fix](planner) do not support UDF without paramter ( #24730 )
...
for example:
CREATE ALIAS FUNCTION f() WITH PARAMETERS() AS now();
2023-09-21 22:23:21 +08:00
c9b2f4cb92
[workload](pipeline) Add cgroup cpu controller ( #24052 )
2023-09-21 21:49:33 +08:00
b8d8cfadfe
[enhancement](delete) avoid null txn state after delete job committed ( #24741 )
2023-09-21 21:21:06 +08:00
58ab25ccaa
Revert "[Feature](merge-on-write)Support ignore mode for merge-on-write unique table ( #21773 )" ( #24731 )
...
This reverts commit 3ee89aea35726197cb7e94bb4f2c36bc9d50da84.
2023-09-21 21:01:28 +08:00
3bd341a1be
[enhancement](schema) Add schema consistency check when add partition ( #24707 )
2023-09-21 20:35:45 +08:00
232226e75a
[Improve](count on index) improve performance for count on index other than match ( #24678 )
2023-09-21 20:30:27 +08:00
6c19e106ad
[fix](rest catalog)support set region for s3 ( #24566 )
...
Use REST Catalog to access S3 and support setting up regions：
```
CREATE CATALOG iceberg_rest_s3 PROPERTIES (
"type"="iceberg",
"iceberg.catalog.type"="rest",
"uri" = "http://127.0.0.1:8181 ",
"s3.endpoint" = "http://127.0.0.1:8010 ",
"s3.access_key" = "admin",
"s3.secret_key" = "password",
"s3.region" = "us-east-1"
);
```
2023-09-21 20:19:43 +08:00
a48b19ceb6
[feature](Outfile) select into outfile supports to export struct/map/array type data to orc file format ( #24350 )
...
We do not support nested complex type in this pr.
2023-09-21 20:15:18 +08:00
aa92c2ddfb
[minor](test) Add some ut for optimizer rule ( #24562 )
...
Add ut for
* ExistsApplyToJoin
* SimplifyDecimalV3Comparison
* SimplifyArithmeticComparisonRule
* SimplifyCastRule
2023-09-21 15:46:35 +08:00
ce57b99ce7
[fix](Nereids) colocate join could not work well on full outer join ( #24700 )
...
in previous PR #22979 , we fix the output deriver of join. But we forgot
to change the util method JoinUtils#shouldColocateJoin to adjust the
change of physical properties derive.
we could not use join distribution type anymore since join could output
any distribute for full outer join.
2023-09-21 15:26:59 +08:00
bcf1806864
[opt](Nereids): GroupExpressionMatching don't need withGroupExpression ( #24684 )
2023-09-21 12:51:44 +08:00
7efaf9d0c8
[chore](add_partition) add some log to investigate problem ( #24193 )
2023-09-21 12:46:18 +08:00
5b590bbfcf
[feat](Nereids) add lambda func array_last and array_first ( #24682 )
2023-09-21 12:23:34 +08:00
a65dbb097c
[enhancement](Nereids) when the sort key is equal to Literal, eliminate sort ( #24669 )
...
when the sort key is equal to Literal, eliminate sort. such as :
filter(a = 1)
|
sort(a)
eliminate this sort(a)
2023-09-21 11:55:53 +08:00
01b7cb8db7
[Bug](materialized-view) fix failed insert into mv meet default null ( #24545 )
...
fix failed insert into mv meet default null
2023-09-21 10:47:29 +08:00
b87ea68720
[Fix](statistics) Fix analyze olap table couldn't get partition names bug ( #24696 )
...
Call getPartitionNames to get all partitions while analyzing for olap table. Couldn't return NULL, otherwise analyze for olap table will do nothing.
2023-09-21 10:28:37 +08:00
