66b14f4db1
[fix](mtmv)fix can not create mtmv all use default value ( #28922 )
2023-12-23 21:27:01 +08:00
5e9e199ca0
[fix](doc) Add the usage example of bos to the documentation of s3 tvf ( #28899 )
2023-12-23 20:47:50 +08:00
de6c7a792e
[fix](chore) update dcheck to avoid core during stress test ( #28895 )
2023-12-23 18:49:57 +08:00
2014396707
[fix](block) add block columns size dcheck ( #28539 )
2023-12-23 15:21:53 +08:00
0a1d9f4cbc
[feature](mtmv)add more test case1 ( #28910 )
2023-12-23 14:39:44 +08:00
623257d02b
[feature](mtmv)MTMV pause and resume ( #28887 )
...
- PAUSE MATERIALIZED VIEW JOB ON mv1
- RESUME MATERIALIZED VIEW JOB ON mv1
- fix when drop db,not drop job
- add lock for one materialized view can only run one task at a time
2023-12-23 14:30:54 +08:00
e51f75e424
[FIX](map)fix map with rowstore table ( #28877 )
2023-12-23 12:11:06 +08:00
4066de375e
[bugfix](scannercore) scanner will core in deconstructor during collect profile ( #28727 )
2023-12-23 11:09:46 +08:00
3d2b4ae244
[fix](mtmv) fix failed to specify the number of buckets when bucket auto ( #28854 )
...
Issue Number: close #xxx
- fix failed to specify the number of buckets when bucket auto
- delete unused SessionVariable
- if mtmv used external table ,check `isMaterializedViewRewriteEnableContainForeignTable`
2023-12-23 09:26:16 +08:00
ae64afd112
[fix](pipelineX) fix cannot runtime obtain profile on pipelineX #28795
2023-12-23 08:24:04 +08:00
43776465d9
[fix](segcompaction) disable segcompaction by default ( #28906 )
...
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com >
2023-12-23 07:43:41 +08:00
e0bb7fa935
[exec](load) change default parallel num from 1 to 8 in no pipeline exec engine ( #28864 )
2023-12-23 00:05:44 +08:00
966766f3b7
[enhancement](broker-load) fix-move-memtable-session-var-for-s3 ( #28894 )
2023-12-22 23:25:06 +08:00
3b830f89a7
[improve](move-memtable) avoid using heavy work pool during append data ( #28745 )
2023-12-22 22:51:30 +08:00
93184bc656
[refactor](test)Refactor workload group/schedule policy test #28888
...
[refactor](test)Refactor workload group/schedule policy test
#28888
2023-12-22 22:18:07 +08:00
726a9b96c2
[enhancement](udf) add prepare function for java-udf ( #28750 )
2023-12-22 22:15:59 +08:00
3eec62e1d1
[fix](multi-catalog)filter impala generated path ( #28786 )
...
file impala generated dir _imapala_insert_staging
2023-12-22 21:58:50 +08:00
f781f0cf24
[improve](load) limit delta writer flush task parallelism ( #28883 )
2023-12-22 21:50:56 +08:00
b1c5747f56
[improve](load) remove extra layer of heavy work pool in tablet_writer_add_block ( #28550 )
2023-12-22 20:10:50 +08:00
18c9ebce95
[improve](move-memtable) tweak load stream flush token num and max tasks ( #28884 )
2023-12-22 20:08:47 +08:00
fa0ad56817
[exec](compress) use FragmentTransmissionCompressionCodec control the exchange compress behavior ( #28818 )
2023-12-22 19:50:57 +08:00
37faf1bb80
[fix](ci) tpch pipeline should not re-load data ( #28874 )
...
* [fix](ci) tpch pipeline should not re-load data
* 2
---------
Co-authored-by: stephen <hello-stephen@qq.com >
2023-12-22 19:50:41 +08:00
b2b209e938
[feature](Nereids) Pull up join from union all ( #28682 )
2023-12-22 19:35:20 +08:00
3ed82bcee2
[Feature](inverted index) add lowercase option for inverted index analyzer ( #28704 )
2023-12-22 18:22:44 +08:00
9e0a2e861c
[pipelineX](refactor) rename functions ( #28846 )
2023-12-22 17:24:39 +08:00
aca8406e31
[refactor](executor)remove scan group #28847
2023-12-22 17:05:50 +08:00
d75300f166
[fix](hash join) fix stack overflow caused by evaluate case expr on huge build block ( #28851 )
2023-12-22 15:45:12 +08:00
cb61a0727f
[bug](sharedscan) Fix shared scan bug ( #28841 )
2023-12-22 15:13:14 +08:00
5420575184
check stats and log memo for ds46 ( #28396 )
2023-12-22 14:50:11 +08:00
7f310cec9c
[opt](sessionVar)show changed sessoin var first #28840
...
“show variables” command list changed vars before not changed vars,
2023-12-22 14:45:33 +08:00
9b67c86219
[optimize](count) optimize pk exact query without reading data ( #28494 )
2023-12-22 14:18:15 +08:00
007f498f3b
(enhance)(InternalQuery) Support to collect profile for intenal query ( #28762 )
2023-12-22 14:03:48 +08:00
d1e1619e89
[feature](mtmv)mtmv partition refresh case ( #28787 )
2023-12-22 14:03:31 +08:00
7710c85904
[fix](stream load)add test case and doc for arrow type of stream load ( #28098 )
...
add test case and doc for arrow type of stream load
2023-12-22 13:18:44 +08:00
f38e11ec4e
[fix](paimon)fix type convert for paimon ( #28774 )
...
fix type convert for paimon
2023-12-22 13:18:25 +08:00
fb52e111c5
[Enhancement](job) No need to query some backends which are not alive. ( #28608 )
...
No need to execute some jobs if backend is not alive
2023-12-22 13:14:30 +08:00
dc786114a2
[Fix](multi-catalog) skip hms events if hms table is not supported. ( #28644 )
...
Co-authored-by: wangxiangyu <wangxiangyu@360shuke.com >
2023-12-22 13:07:58 +08:00
8c59e16f81
[opt](query cancel) optimization for query cancel #28778
2023-12-22 12:48:37 +08:00
e4c7d4d03a
[chore](error msg) print type info when colocate with ddl failed due to type mismatch ( #28773 )
2023-12-22 12:48:20 +08:00
012e66729a
[improvement](executor) Add tvf and regression test for Workload Scheduler ( #28733 )
...
1 Add select workload schedule policy tvf
2 Add reg test
2023-12-22 12:09:51 +08:00
c72ad9b673
[fix](regression) fix regression error of test_compress_type ( #28826 )
2023-12-22 12:08:23 +08:00
83e7235bab
[fix](memory) Add thread asynchronous purge jemalloc dirty pages ( #28655 )
...
jemallctl purge all arena dirty pages may take several seconds, which will block memory GC and cause OOM.
So purge asynchronously in a thread.
2023-12-22 12:05:20 +08:00
13ccfa06a7
[Feature](Variant) Implement variant new sub column access method ( #28484 )
...
* [Feature](Variant) Implement variant new sub column access method
The query SELECT v["a"]["b"] from simple_var WHERE cast(v["a"]["b"] as int) = 1 encompasses three primary testing scenarios:
```
1. A basic test involving the variant data type.
2. A scenario dealing with GitHub event data in the context of a variant.
3. A case related to the TPC-H benchmark using a variant.
```
2023-12-22 11:59:37 +08:00
453e3c18f4
[refactor](buffer) remove download buffer since it is no longer useful ( #28832 )
...
remove download buffer since it is no longer useful
2023-12-22 11:53:31 +08:00
f6b6180462
[Fix](Variant) fix variant predicate rewrite OrToIn with wrong plan ( #28695 )
...
using the name without paths info will lead to wrong In plan, e.g.
```
where cast(v:a as text) = 'hello' or cast(v:b as text) = 'world'
```
will be rewrite to:
```
where cast(v as text) in ('hello', 'world')
``
This is wrong, because they are different slots
2023-12-22 11:51:36 +08:00
0af6bd6390
[fix](group-commit) check if wal need recovery is abnormal ( #28769 )
2023-12-22 11:06:11 +08:00
e2941aa9c8
[improve](config) set mutable and masterOnly in FE config stream_load_default_memtable_on_sink_node ( #28835 )
2023-12-22 10:58:43 +08:00
6962de841d
[fix](test)fix test_create_table test case for nereids ( #28693 )
2023-12-22 10:55:35 +08:00
25dcf40f36
[test](partial update) add complex type regression cases for partial update ( #28758 )
...
NOTE: There's some issue for MAP type to work with row store, so in this PR we don't have cases for MAP type
Will add the support for MAP type in future.
2023-12-22 10:55:24 +08:00
172f68480b
[Enhancement](load) Limit the number of incorrect data drops and add documents ( #27727 )
...
In the load process, if there are problems with the original data, we will store the error data in an error_log file on the disk for subsequent debugging. However, if there are many error data, it will occupy a lot of disk space. Now we want to limit the number of error data that is saved to the disk.
Be familiar with the usage of doris' import function and internal implementation process
Add a new be configuration item load_error_log_limit_bytes = default value 200MB
Use the newly added threshold to limit the amount of data that RuntimeState::append_error_msg_to_file writes to disk
Write regression cases for testing and verification
Co-authored-by: xy720 <22125576+xy720@users.noreply.github.com >
2023-12-22 10:43:18 +08:00
