Commit Graph

8289 Commits

Author SHA1 Message Date
b9928a017b [fix](fe)add variant type in decimalv3's compatibilityMatrix table (#24749) 2023-09-27 10:03:13 +08:00
0dd57b1982 [feature](nereids)support subquey in join condition (#24598) 2023-09-27 10:02:56 +08:00
6d27a016b9 [Improvement](regression-test) add http_stream case (#24930) 2023-09-27 09:55:52 +08:00
2e8f9a6429 [improvement](streamload) QeProcessorImpl#registerQuery should be removed (#24836) 2023-09-27 09:48:27 +08:00
f7f359d36c [fix](analysis)fix use regex determine whether time part exists may cause backtracking (#24882) 2023-09-27 09:26:42 +08:00
f59e69b852 [feature](autobucekt) Add support autobucket min buckets config Add support autobucket (#24920)
Signed-off-by: Jack Drogon <jack.xsuperman@gmail.com>
2023-09-27 08:13:53 +08:00
504ab2d726 [fix](es-catalog) If the returned data is incorrect, it will be directly unified as the index was not found (#24831) 2023-09-26 22:59:38 +08:00
841e6188bf [fix](Nereids): fix roundFloor (#24901)
"roundFloor()" does not consider the issue of carry.
2023-09-26 22:37:42 +08:00
90c5461ad2 [fix](Nereids) let dml work well (#24748)
Co-authored-by: sohardforaname <organic_chemistry@foxmail.com>

TODO:
1. support agg_state type
2. support implicit cast literal exception
3. use nereids execute dml for these regression cases:

- test_agg_state_nereids (for TODO 1)
- test_array_insert_overflow (for TODO 2)
- nereids_p0/json_p0/test_json_load_and_function (for TODO 2)
- nereids_p0/json_p0/test_json_unique_load_and_function (for TODO 2)
- nereids_p0/jsonb_p0/test_jsonb_load_and_function (for TODO 2)
- nereids_p0/jsonb_p0/test_jsonb_unique_load_and_function (for TODO 2)
- json_p0/test_json_load_and_function (for TODO 2)
- json_p0/test_json_unique_load_and_function (for TODO 2)
- jsonb_p0/test_jsonb_load_and_function (for TODO 2)
- jsonb_p0/test_jsonb_unique_load_and_function (for TODO 2)
- test_multi_partition_key (for TODO 2)
2023-09-26 21:08:24 +08:00
a6a0e78f32 [Enhancement](streamload) stream tvf support compress (#24303) 2023-09-26 20:58:20 +08:00
55d1090137 [feature](insert) Support group commit stream load (#24304) 2023-09-26 20:57:02 +08:00
e9eaf49db5 [feature](alter colocate group) Support alter colocate group replica allocation (#23320) 2023-09-26 19:55:25 +08:00
77e864df12 [enhancement](delete) use column id in delete push task instead of column name (#24549) 2023-09-26 19:54:55 +08:00
04bf9bce54 [fix](planner)update explode slot's nullable info in analyze phase (#24879) 2023-09-26 18:14:04 +08:00
082bcd820b [feature](insert) Support wal for group commit insert (#23053) 2023-09-26 14:46:24 +08:00
983f948ec4 [fix](Nereids): fix SimplifyComparisonPredicate (#24899)
#16061 exist BUG, it causes wrong cast
2023-09-26 14:28:03 +08:00
43433c3fab [improvement](processlist) Add more information for show processlist (#24000) 2023-09-26 14:25:42 +08:00
6890e5bf0f [test](Nereids): add test for SimplifyComparisonPredicate (#24865) 2023-09-26 12:50:31 +08:00
eb9d68a640 [opt](Nereids) add json literal for constant folding (#24878) 2023-09-26 11:37:53 +08:00
1f8e0b48bc [fix](S3)delete main function because hardcoded ip is not safe (#24872) 2023-09-26 10:49:16 +08:00
33630e1c4d [fix](multi-catalog) SecureRandom is used to replace Random in FederationBackendPolicy to enhance security (#24860) 2023-09-26 10:44:22 +08:00
e887653dde [fix](auth) Remove printing passwords in the log (#24866) 2023-09-26 10:17:44 +08:00
e4c0c98efa [fix](Nereids): round microsecond when specify scale of microsecond (#24854) 2023-09-26 10:11:53 +08:00
b9472bf5d3 [fix](LOG)change printStackTracker to LOG (#24884) 2023-09-26 08:28:53 +08:00
60ce1ed8a3 [improvement](tablet clone) fix balanced new replica will be removed when load txn continuously and none-stop (#24845) 2023-09-25 22:51:08 +08:00
8d4fd76a16 [Feature](StreamLoad2PC) Support commit and abort streamload2PC by label (#24613) 2023-09-25 22:21:27 +08:00
158570787a [fix](multi-catalog)The external catalog does not have verification parameters, causing an error to be reported when creating the catalog. (#24393) 2023-09-25 21:35:52 +08:00
12bee54cf7 [fix](Nereids) function implict cast for json is not work well (#24852)
for sql like:

SELECT JSONB_EXTRACT('{"k1":"v31","k2":300}','$.k1');

the result should be

+------------------------------------------------+
| jsonb_extract('{"k1":"v31","k2":300}', '$.k1') |
+------------------------------------------------+
| "v31"                                          |
+------------------------------------------------+

but curent result is

+------------------------------------------------+
| jsonb_extract('{"k1":"v31","k2":300}', '$.k1') |
+------------------------------------------------+
| <null>                                         |
+------------------------------------------------+
2023-09-25 21:10:04 +08:00
6502da8917 [bugfix](restore) add partition id into convert_rowset_ids() (#24834) 2023-09-25 20:07:24 +08:00
9dff620471 [feature](binlog) add truncate table binlog (#24868)
Signed-off-by: Jack Drogon <jack.xsuperman@gmail.com>
2023-09-25 19:53:42 +08:00
3e686f4306 [feature](javaudf)support no input java udf (#24457)
* support no input java udf

* add license
2023-09-25 18:56:44 +08:00
8679095e5c [feature](debug) support debug point used in debug code (#24502) 2023-09-25 17:56:12 +08:00
90791f0b19 [fix](nereids)fix bug of exists subquery with limit clause (#24630)
create table t1(c1 int, c2 int);
create table t2(c1 int, c2 int);
insert into t1 values (1,1);
insert into t2 values (1,1);

select * from t1 where exists (select * from t2 where t1.c1 = t2.c1 limit 0);

the result should be empty set.
2023-09-25 17:15:08 +08:00
60af62b41e [enhancement](Nereids) remove deriveStats jobs for some groupExpression (#24835)
Don't call DeriveStatJob for follow group expressions:

- the group expression that is generated by the joinCommute rule
- the group expression that is generated by the implementation rule without creating a new group
2023-09-25 17:14:16 +08:00
xfz
1b95ce1d93 [feature](json-function) add json_insert, json_replace, json_set functions (#24384)
[feature](json-function) add three json funcitons
2023-09-25 12:52:29 +08:00
82b393314c [enhancement](Nereids) check unimplemented func in nereids (#24766) 2023-09-25 12:45:11 +08:00
2d0ac7117a [enhancement](show-frontends-disks) Add disk info corresponding to deploy dir in frontend disk info (#24726) 2023-09-25 11:14:03 +08:00
ec93ea22b3 [bugfix](Coordinator) The error msg hide host info exclude localhost (#24817)
Exception information is incomplete when localhost exists in the log.
2023-09-25 09:45:00 +08:00
8fd27a852c [fix](transaction) fix profile null exception cause txn stuck (#24813) 2023-09-24 23:46:38 +08:00
16aee69911 [improvement](tablet schedule) disk balance skip cooldown storage (#24580) 2023-09-24 23:09:29 +08:00
900b06d14e [improvement](disk balancer) reduce disk balance sensitivity (#24637) 2023-09-24 22:02:53 +08:00
ece2e3cd70 [fix](nereids) support cte rf pushdown for exttable (#24806)
Currently, cte inside rf pushdown doesn't support external table, this pr open this restriction.
And since rf for schema scan, for example tables under information_schema, currently isn't supported by be, this pr also check and prevent to generate rf on them.
2023-09-23 23:00:05 +08:00
ce8dfd3561 [Chore](dependency)grpc library dependencies are unified (#24794) 2023-09-23 14:29:34 +08:00
bf07415ff0 [fix](es catalog) remove _default_ field when parsing mapping. (#24781) 2023-09-23 10:58:55 +08:00
8ca300f70b [Opt](pipeline) disable shared scan in key point search (#24797) 2023-09-22 22:46:11 +08:00
b86f09418f [chore](build) Fix the FE build on CentOS 6 (#24798)
Using grpc-java whose version is newer than 1.34.0 will break the build on CentOS 6 due to the obsolete GLIBC.
2023-09-22 19:58:12 +08:00
35c0697a60 [fix](planner)SelectStmt's constructor should initialize originSelectList member (#24755)
resetSelectList method will use originSelectList to recover the origin select list. If the originSelectList is lost in constructor, the resetSelectList will fail to recover and make the analyze process fail.
2023-09-22 19:31:29 +08:00
eb9d334b55 [opt](nereids) enable runtime filter prune by default (#24717)
add stats back to physical plan in post processors. stats are used by rf-prune
set enable_runtime_filter_prune true by default
2023-09-22 19:11:10 +08:00
c943a05065 [fix](stats) Fix data size calculation of auto sample (#24672)
1. Fix data size calculation of auto sample, before this pr, the data size is include all the replicas
2. Move some auto analyze related options to global session variable
3. Add some logs
2023-09-22 18:12:39 +08:00
b34a4779e5 [feat](optimizer) calculate stats health based on real updated rows count (#24599)
Use actual load rows since last analyze rather than delta of total row count.
2023-09-22 18:12:13 +08:00