Commit Graph

15785 Commits

Author SHA1 Message Date
dfbf082e06 [fix](merge-on-write) migration may cause duplicate keys for mow table (#28923) 2023-12-23 23:37:00 +08:00
1d8822b2b7 [fix](paimon)fix like predicate (#28803)
fix like predict
2023-12-23 22:25:55 +08:00
96d4778f2e [fix](parquet) the end offset of column chunk may be wrong in parquet metadata (#28891) 2023-12-23 22:21:04 +08:00
66b14f4db1 [fix](mtmv)fix can not create mtmv all use default value (#28922) 2023-12-23 21:27:01 +08:00
5e9e199ca0 [fix](doc) Add the usage example of bos to the documentation of s3 tvf (#28899) 2023-12-23 20:47:50 +08:00
de6c7a792e [fix](chore) update dcheck to avoid core during stress test (#28895) 2023-12-23 18:49:57 +08:00
2014396707 [fix](block) add block columns size dcheck (#28539) 2023-12-23 15:21:53 +08:00
0a1d9f4cbc [feature](mtmv)add more test case1 (#28910) 2023-12-23 14:39:44 +08:00
623257d02b [feature](mtmv)MTMV pause and resume (#28887)
- PAUSE MATERIALIZED VIEW JOB ON mv1
- RESUME MATERIALIZED VIEW JOB ON mv1
- fix when drop db,not drop job
- add lock for one materialized view can only run one task at a time
2023-12-23 14:30:54 +08:00
e51f75e424 [FIX](map)fix map with rowstore table (#28877) 2023-12-23 12:11:06 +08:00
4066de375e [bugfix](scannercore) scanner will core in deconstructor during collect profile (#28727) 2023-12-23 11:09:46 +08:00
3d2b4ae244 [fix](mtmv) fix failed to specify the number of buckets when bucket auto (#28854)
Issue Number: close #xxx

- fix failed to specify the number of buckets when bucket auto
- delete unused SessionVariable
- if mtmv used external table ,check `isMaterializedViewRewriteEnableContainForeignTable`
2023-12-23 09:26:16 +08:00
ae64afd112 [fix](pipelineX) fix cannot runtime obtain profile on pipelineX #28795 2023-12-23 08:24:04 +08:00
43776465d9 [fix](segcompaction) disable segcompaction by default (#28906)
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
2023-12-23 07:43:41 +08:00
e0bb7fa935 [exec](load) change default parallel num from 1 to 8 in no pipeline exec engine (#28864) 2023-12-23 00:05:44 +08:00
966766f3b7 [enhancement](broker-load) fix-move-memtable-session-var-for-s3 (#28894) 2023-12-22 23:25:06 +08:00
3b830f89a7 [improve](move-memtable) avoid using heavy work pool during append data (#28745) 2023-12-22 22:51:30 +08:00
93184bc656 [refactor](test)Refactor workload group/schedule policy test #28888
[refactor](test)Refactor workload group/schedule policy test
#28888
2023-12-22 22:18:07 +08:00
726a9b96c2 [enhancement](udf) add prepare function for java-udf (#28750) 2023-12-22 22:15:59 +08:00
3eec62e1d1 [fix](multi-catalog)filter impala generated path (#28786)
file impala generated dir _imapala_insert_staging
2023-12-22 21:58:50 +08:00
f781f0cf24 [improve](load) limit delta writer flush task parallelism (#28883) 2023-12-22 21:50:56 +08:00
b1c5747f56 [improve](load) remove extra layer of heavy work pool in tablet_writer_add_block (#28550) 2023-12-22 20:10:50 +08:00
18c9ebce95 [improve](move-memtable) tweak load stream flush token num and max tasks (#28884) 2023-12-22 20:08:47 +08:00
fa0ad56817 [exec](compress) use FragmentTransmissionCompressionCodec control the exchange compress behavior (#28818) 2023-12-22 19:50:57 +08:00
37faf1bb80 [fix](ci) tpch pipeline should not re-load data (#28874)
* [fix](ci) tpch pipeline should not re-load data

* 2

---------

Co-authored-by: stephen <hello-stephen@qq.com>
2023-12-22 19:50:41 +08:00
b2b209e938 [feature](Nereids) Pull up join from union all (#28682) 2023-12-22 19:35:20 +08:00
3ed82bcee2 [Feature](inverted index) add lowercase option for inverted index analyzer (#28704) 2023-12-22 18:22:44 +08:00
9e0a2e861c [pipelineX](refactor) rename functions (#28846) 2023-12-22 17:24:39 +08:00
aca8406e31 [refactor](executor)remove scan group #28847 2023-12-22 17:05:50 +08:00
d75300f166 [fix](hash join) fix stack overflow caused by evaluate case expr on huge build block (#28851) 2023-12-22 15:45:12 +08:00
cb61a0727f [bug](sharedscan) Fix shared scan bug (#28841) 2023-12-22 15:13:14 +08:00
5420575184 check stats and log memo for ds46 (#28396) 2023-12-22 14:50:11 +08:00
7f310cec9c [opt](sessionVar)show changed sessoin var first #28840
“show variables” command list changed vars before not changed vars,
2023-12-22 14:45:33 +08:00
9b67c86219 [optimize](count) optimize pk exact query without reading data (#28494) 2023-12-22 14:18:15 +08:00
007f498f3b (enhance)(InternalQuery) Support to collect profile for intenal query (#28762) 2023-12-22 14:03:48 +08:00
d1e1619e89 [feature](mtmv)mtmv partition refresh case (#28787) 2023-12-22 14:03:31 +08:00
7710c85904 [fix](stream load)add test case and doc for arrow type of stream load (#28098)
add test case and doc for arrow type of stream load
2023-12-22 13:18:44 +08:00
f38e11ec4e [fix](paimon)fix type convert for paimon (#28774)
fix type convert for paimon
2023-12-22 13:18:25 +08:00
fb52e111c5 [Enhancement](job) No need to query some backends which are not alive. (#28608)
No need to execute some jobs if backend is not alive
2023-12-22 13:14:30 +08:00
dc786114a2 [Fix](multi-catalog) skip hms events if hms table is not supported. (#28644)
Co-authored-by: wangxiangyu <wangxiangyu@360shuke.com>
2023-12-22 13:07:58 +08:00
8c59e16f81 [opt](query cancel) optimization for query cancel #28778 2023-12-22 12:48:37 +08:00
e4c7d4d03a [chore](error msg) print type info when colocate with ddl failed due to type mismatch (#28773) 2023-12-22 12:48:20 +08:00
012e66729a [improvement](executor) Add tvf and regression test for Workload Scheduler (#28733)
1 Add select workload schedule policy tvf
2 Add reg test
2023-12-22 12:09:51 +08:00
c72ad9b673 [fix](regression) fix regression error of test_compress_type (#28826) 2023-12-22 12:08:23 +08:00
83e7235bab [fix](memory) Add thread asynchronous purge jemalloc dirty pages (#28655)
jemallctl purge all arena dirty pages may take several seconds, which will block memory GC and cause OOM.
So purge asynchronously in a thread.
2023-12-22 12:05:20 +08:00
13ccfa06a7 [Feature](Variant) Implement variant new sub column access method (#28484)
* [Feature](Variant) Implement variant new sub column access method

The query SELECT v["a"]["b"] from simple_var WHERE cast(v["a"]["b"] as int) = 1 encompasses three primary testing scenarios:

```
1. A basic test involving the variant data type.
2. A scenario dealing with GitHub event data in the context of a variant.
3. A case related to the TPC-H benchmark using a variant.
```
2023-12-22 11:59:37 +08:00
453e3c18f4 [refactor](buffer) remove download buffer since it is no longer useful (#28832)
remove download buffer since it is no longer useful
2023-12-22 11:53:31 +08:00
f6b6180462 [Fix](Variant) fix variant predicate rewrite OrToIn with wrong plan (#28695)
using the name without paths info will lead to wrong In plan, e.g.
```
where cast(v:a as text) = 'hello' or cast(v:b as text) = 'world'
```
will be rewrite to:
```
where cast(v as text) in ('hello', 'world')
``
This is wrong, because they are different slots
2023-12-22 11:51:36 +08:00
0af6bd6390 [fix](group-commit) check if wal need recovery is abnormal (#28769) 2023-12-22 11:06:11 +08:00
e2941aa9c8 [improve](config) set mutable and masterOnly in FE config stream_load_default_memtable_on_sink_node (#28835) 2023-12-22 10:58:43 +08:00