Commit Graph

8289 Commits

Author SHA1 Message Date
2dc7d82aff [Chore](Job)print log before task execute (#28962) 2023-12-25 16:55:59 +08:00
17917a0adb [feature](load) enable memtable on sink node by default (#28963) 2023-12-25 16:27:16 +08:00
29d3d5e433 [Test](Job)Add test case (#28481) 2023-12-25 15:12:14 +08:00
1d984e0ebb return residual expr of join (#28760) 2023-12-25 12:53:14 +08:00
e9e1e2894b [performance](variant) support topn 2phase read for variant column (#28318)
[performance](variant) support topn 2phase read for variant column
2023-12-25 11:50:41 +08:00
48f8f8f2ad [fix](nereids)group by expr may be bound twice in bind agg slot (#28771) 2023-12-25 11:23:44 +08:00
c53611dcb3 [feature](nereids)support decimalv2 (#28726) 2023-12-25 11:23:27 +08:00
99755920e2 [fix](nereids)subquery unnest need handle subquery in Not expr correnctly (#28713) 2023-12-25 11:23:00 +08:00
b49671b31e [feature](api) add profile text api (#28697) 2023-12-25 10:17:09 +08:00
d8cb4da73d [chore](prompt) Optimize tablet and replica prompt by pointing out what the numbers mean (#28925) 2023-12-24 22:29:24 +08:00
2f7deb3dea [enhancement](bulk-load) cancel loading tasks directly without retrying when timeout exceeded (#28666) 2023-12-24 21:57:31 +08:00
1e44a4b145 [opt](nereids) convert or to inpredicate and optimize inpredicate partition prune (#28316) 2023-12-24 21:42:03 +08:00
145683ccdb [improvement](group commit) make get column function more reliable when replaying wal (#28900) 2023-12-24 21:17:39 +08:00
b8de5cfc36 [fix] (nereids) Catch exception when mv fail and fix the npe (#28932) 2023-12-24 20:41:24 +08:00
6ea6ff5dee [fix](nereids) Fix query mv rewrite fail when mv cache build quickly (#28876) 2023-12-24 09:00:49 +08:00
5505fa3755 [Fix](statistics) Fix partition name NPE and sample for all table during auto analyze (#28916)
Fix partition name NPE and sample for all table during auto analyze.
Sample for all tables because getData may have latency, which may cause full analyze a huge table and use too much resource. Sample for all tables to avoid this. Will improve the strategy later.
2023-12-24 01:49:49 +08:00
13a3550d8e [nereids] fix join fd computing bug (#28849) 2023-12-24 01:41:43 +08:00
37777dcdb5 [fix](mtmv) Related partition exclude null generate column when increment build materialized view (#28855)
Infer partition column by materialized view partition column, exclude null generate column in join when increment build materialized view
2023-12-24 01:37:55 +08:00
1d8822b2b7 [fix](paimon)fix like predicate (#28803)
fix like predict
2023-12-23 22:25:55 +08:00
66b14f4db1 [fix](mtmv)fix can not create mtmv all use default value (#28922) 2023-12-23 21:27:01 +08:00
0a1d9f4cbc [feature](mtmv)add more test case1 (#28910) 2023-12-23 14:39:44 +08:00
623257d02b [feature](mtmv)MTMV pause and resume (#28887)
- PAUSE MATERIALIZED VIEW JOB ON mv1
- RESUME MATERIALIZED VIEW JOB ON mv1
- fix when drop db,not drop job
- add lock for one materialized view can only run one task at a time
2023-12-23 14:30:54 +08:00
3d2b4ae244 [fix](mtmv) fix failed to specify the number of buckets when bucket auto (#28854)
Issue Number: close #xxx

- fix failed to specify the number of buckets when bucket auto
- delete unused SessionVariable
- if mtmv used external table ,check `isMaterializedViewRewriteEnableContainForeignTable`
2023-12-23 09:26:16 +08:00
ae64afd112 [fix](pipelineX) fix cannot runtime obtain profile on pipelineX #28795 2023-12-23 08:24:04 +08:00
e0bb7fa935 [exec](load) change default parallel num from 1 to 8 in no pipeline exec engine (#28864) 2023-12-23 00:05:44 +08:00
966766f3b7 [enhancement](broker-load) fix-move-memtable-session-var-for-s3 (#28894) 2023-12-22 23:25:06 +08:00
726a9b96c2 [enhancement](udf) add prepare function for java-udf (#28750) 2023-12-22 22:15:59 +08:00
3eec62e1d1 [fix](multi-catalog)filter impala generated path (#28786)
file impala generated dir _imapala_insert_staging
2023-12-22 21:58:50 +08:00
fa0ad56817 [exec](compress) use FragmentTransmissionCompressionCodec control the exchange compress behavior (#28818) 2023-12-22 19:50:57 +08:00
b2b209e938 [feature](Nereids) Pull up join from union all (#28682) 2023-12-22 19:35:20 +08:00
3ed82bcee2 [Feature](inverted index) add lowercase option for inverted index analyzer (#28704) 2023-12-22 18:22:44 +08:00
cb61a0727f [bug](sharedscan) Fix shared scan bug (#28841) 2023-12-22 15:13:14 +08:00
7f310cec9c [opt](sessionVar)show changed sessoin var first #28840
“show variables” command list changed vars before not changed vars,
2023-12-22 14:45:33 +08:00
007f498f3b (enhance)(InternalQuery) Support to collect profile for intenal query (#28762) 2023-12-22 14:03:48 +08:00
f38e11ec4e [fix](paimon)fix type convert for paimon (#28774)
fix type convert for paimon
2023-12-22 13:18:25 +08:00
fb52e111c5 [Enhancement](job) No need to query some backends which are not alive. (#28608)
No need to execute some jobs if backend is not alive
2023-12-22 13:14:30 +08:00
dc786114a2 [Fix](multi-catalog) skip hms events if hms table is not supported. (#28644)
Co-authored-by: wangxiangyu <wangxiangyu@360shuke.com>
2023-12-22 13:07:58 +08:00
e4c7d4d03a [chore](error msg) print type info when colocate with ddl failed due to type mismatch (#28773) 2023-12-22 12:48:20 +08:00
012e66729a [improvement](executor) Add tvf and regression test for Workload Scheduler (#28733)
1 Add select workload schedule policy tvf
2 Add reg test
2023-12-22 12:09:51 +08:00
13ccfa06a7 [Feature](Variant) Implement variant new sub column access method (#28484)
* [Feature](Variant) Implement variant new sub column access method

The query SELECT v["a"]["b"] from simple_var WHERE cast(v["a"]["b"] as int) = 1 encompasses three primary testing scenarios:

```
1. A basic test involving the variant data type.
2. A scenario dealing with GitHub event data in the context of a variant.
3. A case related to the TPC-H benchmark using a variant.
```
2023-12-22 11:59:37 +08:00
f6b6180462 [Fix](Variant) fix variant predicate rewrite OrToIn with wrong plan (#28695)
using the name without paths info will lead to wrong In plan, e.g.
```
where cast(v:a as text) = 'hello' or cast(v:b as text) = 'world'
```
will be rewrite to:
```
where cast(v as text) in ('hello', 'world')
``
This is wrong, because they are different slots
2023-12-22 11:51:36 +08:00
0af6bd6390 [fix](group-commit) check if wal need recovery is abnormal (#28769) 2023-12-22 11:06:11 +08:00
e2941aa9c8 [improve](config) set mutable and masterOnly in FE config stream_load_default_memtable_on_sink_node (#28835) 2023-12-22 10:58:43 +08:00
c1457f97a4 [improvement](external catalog)Change log level to debug for getRowCountFromExternalSource. (#28801) 2023-12-21 22:09:50 +08:00
65e3da255f [enhancement](audit-log) add status code and error msg to audit log for proxy stmt (#27607) 2023-12-21 21:16:36 +08:00
e51e94fdec [fix](Nereids) join order is not right after sql parsing (#28721)
for sql
```
t1, t2 join t3
```

we should generate plan like:
```
t1 join (t2 join t3)
```

but we generate:
```
(t1 join t2) join t3
```
to follow legancy planner.
2023-12-21 20:31:40 +08:00
0070909d30 [fix](group commit)Fix the issue of duplicate addition of wal path when encouter exception (#28691) 2023-12-21 20:27:33 +08:00
ba4033ad2e [opt](Nereids) improve Shape check cases (#28124)
* tmplate update
* log tpcds stats when check shape
2023-12-21 20:19:54 +08:00
619d500333 [bug](pipelineX) Fix pipelineX bug on multiple BE (#28792) 2023-12-21 19:15:52 +08:00
a16680cf91 [fix](planner)fix bug of bound conjunct to wrong tuple (#28811)
this fix bug introduced by #28656
2023-12-21 19:15:36 +08:00