d127d67ebe
Revert "[fix](csv-reader) fix column split error when there is escape character ( #34364 )"
...
This reverts commit 971e10a9db782c9986b20e1209468e4d7aeedf71.
2024-05-07 13:36:11 +08:00
4ecc3edc21
[test](hive)revert hive container to host mode ( #34322 )
...
Revert hive container to host mode to fix pipeline problem
2024-05-07 10:36:01 +08:00
63cd632abe
Revert "[fix](statistics) Use column update rows to decide min/max stats are valid or not ( #34263 )"
...
This reverts commit 8db4d48731688354d6ee3ae22e02041419ca73e0.
2024-05-07 08:09:37 +08:00
a33715bc1c
[fix](partial update) only unique table with MOW insert with target columns can consider be a partial update ( #33656 )
...
* [fix](partial update) only unique table with MOW insert with target columns can consider be a partial update
Signed-off-by: nextdreamblue <zxw520blue1@163.com >
* fix 1
Signed-off-by: nextdreamblue <zxw520blue1@163.com >
---------
Signed-off-by: nextdreamblue <zxw520blue1@163.com >
2024-05-07 07:53:25 +08:00
92dc8ed718
[opt](mtmv) Add enable materialized view nest rewrite switch ( #34197 )
...
* [opt](mtmv) Add enable materialized view nest rewrite switch
* fix ut
* fix ut2
2024-05-07 07:51:18 +08:00
818022cadf
[Fix](MethodName) fix method issue #34178
2024-05-07 07:50:54 +08:00
2b5bac3b60
[enhance](serde) expr serde support gson ( #34180 )
...
In the future, it can be easier to change to Expression to avoid metadata compatibility issues
2024-05-07 07:50:27 +08:00
e840102e99
[Feat](nereids)nereids support create table like ( #34025 )
...
nereids support create table like statement.
e.g. CREATE TABLE test1.table2 LIKE test1.table1
2024-05-07 07:50:19 +08:00
8f40882701
[fix](nereids) disable PROJECT_OTHER_JOIN_CONDITION rule if bitmap filter is enabled. #34189
...
this pr is a quick solution, but not complete.
runtime filter on NestLoopJoin suffers this bug even without PROJECT_OTHER_JOIN_CONDITION rule.
for example, when enable Min_Max Runtime filter, the target Expression is n_regionkey, but it should be "n_regionkey - 28"
explain
select n_nationkey, nrkey
from (select n_regionkey -28 nrkey, n_nationkey from nation) T
join region on nrkey > r_regionkey;
we will refactor RuntimeFilterGenerator to completely solve this issue in following pr.
2024-05-07 07:49:35 +08:00
74029f56d4
[BugFix](TabletInvertedIndex) fix replica not found in TabletInvertedIndex ( #34117 )
...
* fix replica not found in TabletInvertedIndex
2024-05-07 07:48:13 +08:00
fe7d2b8159
[Fix](nereids) ignore slot implements SlotNotFromChildren when check the slot from children in NormalizeAggregate ( #34171 )
2024-05-07 07:48:05 +08:00
f90c2f6401
fix syntax error for CreateTableLikeStmt with partition properties ( #34187 )
...
fix syntax error for CreateTableLikeStmt with partition properties
2024-05-07 07:47:39 +08:00
8ee7bc430d
[fix](Nereids) should derive stats asap to avoid npe ( #34238 )
...
we do derive stats job eager to avoid un derive stats due to merge group and optimize group
consider:
we have two groups burned by order: G1 and G2
then we have job by order derive G2, optimize group expression in G2,
derive G1, optimize group expression in G1
if G1 merged into G2, then we maybe generated job optimize group G2 before derive G1
in this case, we will do get stats from G1's child before derive G1's child stats
then we will meet NPE in CostModel.
2024-05-07 07:47:07 +08:00
84396dbb39
[enhance](mtmv) When specifying a data model in a materialized view, it is compatible with the syntax of an OLAP table ( #34231 )
2024-05-07 07:46:30 +08:00
e66dcd0e72
[Bug](materialized-view) change nvl to ifnull when create mv ( #34272 )
...
change nvl to ifnull when create mv
2024-05-07 07:45:33 +08:00
e9064d1b94
[fix](Nereids) topn should not inherit logical properties when repace child ( #34282 )
2024-05-07 07:44:36 +08:00
a391cf6bfe
[fix](Nereids) rewritten mv should check output set and should not return null ( #34288 )
...
1. we should check output set, since we will remove top project and
result output size will diff with its child output size if there are
dup slot in result list
2. should not return null, instead we should return rewritten plan
itself, because we will use return result in many place and do not
check null at all
2024-05-07 07:44:16 +08:00
9d0d7293f0
[fix](json) fix be crash while load json data ( #34283 )
2024-05-07 07:42:53 +08:00
8db4d48731
[fix](statistics) Use column update rows to decide min/max stats are valid or not ( #34263 )
...
This is a following pr of #33685
After #33703 merged, need to check update rows in column level instead of table level.
2024-05-07 07:41:28 +08:00
971e10a9db
[fix](csv-reader) fix column split error when there is escape character ( #34364 )
2024-05-07 07:38:35 +08:00
ad35968236
[Fix](Job)Job repaly logic error ( #34378 )
2024-05-07 07:37:14 +08:00
3fd3dfe16f
[Feat](Job) Job supports task execution statistics ( #34109 )
...
* Support statistics
* - Fix Failed task not showing up in the task list
- Task metadata add jobName
- Fix Finished job clear time error
- Job metadata add successCount, failedCount, totalTaskCount
* add test
2024-05-07 07:36:54 +08:00
956ae2f83d
[opt](Nereids) let behavior of function char same with legacy planner ( #34415 )
...
1. first argument must be string like literal
2. only support utf-8 charset
2024-05-07 07:34:34 +08:00
8fdfbcb3c4
Revert "[Opt](func) opt the percentile func performance ( #34373 ) ( #34416 )"
...
This reverts commit 509ae425e416b4779ae94eab9c2b21f9850e03c3.
2024-05-07 07:23:48 +08:00
2d4da7d177
[fix](kerberos)enable hadoop auto renew tgt ( #34439 )
2024-05-07 00:36:20 +08:00
e19d57261c
[improvement](spill) improve cancel ( #34451 )
...
* [improvement](spill) improve cancel
* fix
2024-05-07 00:07:20 +08:00
a81beb19c2
[fix](load) fix repeatedly open tablets_channel when tablets_channel already cancelled ( #34442 )
2024-05-06 23:15:33 +08:00
f7900b53ce
[enhancement](function) floor/ceil/round/round_bankers can use column as scale argument ( #34391 )
2024-05-06 22:18:36 +08:00
c22f42121b
[fix](compaction test) show single replica compaction status and fix test ( #33076 ) ( #34285 ) ( #34438 )
2024-05-06 21:00:34 +08:00
b7b843d944
[fix](load) acquire latest token instead of oldest token in TokenManager ( #34424 )
...
* [fix](load) acquire latest token instead of oldest token
* fixup
2024-05-06 20:19:36 +08:00
aa156f0781
[opt](memory) BE memory info compatible with Cgroup ( #34262 )
2024-05-06 20:11:20 +08:00
11ca738261
[fix](memory) Fix thread context init in MacOS and not use memory tracker ( #34125 )
2024-05-06 20:11:20 +08:00
509ae425e4
[Opt](func) opt the percentile func performance ( #34373 ) ( #34416 )
2024-05-06 20:10:35 +08:00
3cb0deae9c
[opt](ranger) modify and enhance the feature of ranger access controller ( #34392 ) ( #34426 )
...
bp #34392
2024-05-06 17:08:47 +08:00
ab5ee81811
[fix](memory) Fix page cache memory tracker consumption in prune ( #34320 )
2024-05-06 12:53:11 +08:00
7ae5de316b
[feature](Nereids) support set and use statement syntax only ( #33979 ) ( #34409 )
...
pick from master #33979
commit id 65fb7d43b7e838c48502d4e8a69e2541dc73aa88
This PR:
1. add a new Command type: UnsupportedCommand to handle the statement only support parse but could not execute.
2. support syntax about set and use
3. add keyword VAULT to follow legacy planner
TODO
1. support all statment syntax in Nereids
2024-05-06 11:36:01 +08:00
d4c2f70673
[fix](regression_test) failed in distinct_streaming_agg ( #34404 )
2024-05-06 10:11:39 +08:00
85ae773996
[fix](spill) incorrect revocable mem size of hash join ( #34379 )
2024-05-06 06:53:12 +08:00
7248420cfd
[chore](session_variable) Add 'data_queue_max_blocks' to prevent the DataQueue from occupying too much memory. ( #34017 ) ( #34395 )
2024-05-05 21:20:33 +08:00
c3096cabe2
[Fix](executor)normal group not auth #34377
2024-05-02 15:17:19 +08:00
0d106fe4c2
[Bug](runtime-filter) release rf count dependency when query canceled ( #34367 )
...
* release rf count dependency when query canceled
* update
* update
2024-05-02 09:56:17 +08:00
8da260ee0d
[fix](hdfs)read 'fs.defaultFS' from core-site.xml for hdfs load which has no default fs ( #34217 ) ( #34372 )
...
bp #34217
Co-authored-by: slothever <18522955+wsjz@users.noreply.github.com >
2024-05-01 00:31:49 +08:00
581e168ee1
[Fix](executor)Fix normal workload group alter may failed #34356
2024-04-30 22:17:40 +08:00
5fc1f11cf1
[improvement](hive)add the queryid to the temporary file path ( #34278 ) ( #34368 )
...
bp #34278
Co-authored-by: wuwenchi <wuwenchihdu@hotmail.com >
2024-04-30 22:00:05 +08:00
8abd136ba2
[Improvement](executor)Refactor Workload group memory GC ( #33797 )
...
* just gc group's overcommit query when minor gc
* add process usage
2024-04-30 19:34:31 +08:00
50855f01c7
[fix](nereids) when runtimefilter target is null, skip the rf #34358
2024-04-30 18:48:50 +08:00
35f8563a75
[feature](iceberg) support iceberg equality delete ( #34223 ) ( #34327 )
...
bp #34223
Co-authored-by: Ashin Gau <AshinGau@users.noreply.github.com >
2024-04-30 11:51:29 +08:00
7d77fd0286
[fix](profile) Fix reporting the profile while building the pipeline profile. ( #34215 ) ( #34326 )
2024-04-30 11:38:03 +08:00
843c89f109
[fix] fix nullptr when clear cache due to move ( #34323 )
2024-04-30 09:52:41 +08:00
6f873c5907
[improvement](join) Avoid merging blocks more than once on the build side ( #34291 )
2024-04-30 08:37:53 +08:00
