a8d8c6a271
[fix](file-writer) opt s3 file writer and fix empty file related issue #28983 #30703 #31169 ( #31213 )
...
* (feature)(cloud) Use dynamic allocator instead of static buffer pool for better elasticity. (#28983 )
* [fix](outfile) Fix unable to export empty data (#30703 )
Issue Number: close #30600
Fix unable to export empty data to hdfs / S3, this behavior is inconsistent with version 1.2.7,
version 1.2.7 can export empty data to hdfs/ S3, and there will be exported files on S3/HDFS.
* [fix](file-writer) avoid empty file for segment writer (#31169 )
---------
Co-authored-by: AlexYue <yj976240184@gmail.com >
Co-authored-by: zxealous <zhouchangyue@baidu.com >
2024-02-21 16:48:54 +08:00
33925a738d
[improvement](schemachange) add schema change log ( #31145 ) ( #31203 )
2024-02-21 14:15:02 +08:00
c27692fb3b
[Enhancement](jdbc catalog) Add security check on driver when creating Jdbc Catalog ( #31153 )
2024-02-21 13:53:40 +08:00
a4f9eec810
[chore](third-party) Link protobuf with zlib statically ( #31173 )
...
We should link protobuf with libz.a explicitly, otherwise protoc may not be executable due to the missing libz.1.dylib.
2024-02-21 13:53:39 +08:00
f121ee907f
[fix](invert index) fix the inaccurate rows_inverted_index_filtered in the profile ( #31158 )
2024-02-21 13:53:39 +08:00
728b12cfdf
[regression-test](point query) make test stable ( #31140 )
2024-02-21 13:53:32 +08:00
278b232e76
[Bug](json reader) object should stop processing when encounter error ( #31159 )
...
If DATA_QUALITY_ERROR encountered we should stop processing this document any more.Otherwise there will be UB in simdjson.
2024-02-21 13:53:32 +08:00
8b6d6d0165
[pipelineX](refactor) refactor streaming agg structure ( #31151 )
2024-02-21 13:53:32 +08:00
cd7230885f
[fix](nereids)push more than one runtime filters into cte ( #30901 )
...
* push rf into cte, used by tpcds95
2024-02-21 13:53:32 +08:00
c734e79d14
[docs](docs) fix doc MySQL Load format ( #30182 )
...
* chore: fix doc MySql Load format
* chore: modify mysql-load-manual.md
* chore: modify to MySQL
2024-02-21 13:53:32 +08:00
1e3968fe7e
[fix](group_commit) Need to wait wal to be deleted when creating MaterializedView ( #30956 )
2024-02-21 13:53:19 +08:00
ccac4208e1
[fix](regression test) fix test_dynamic_partition_with_alter #31154
2024-02-21 13:53:19 +08:00
69c7750db2
[Improment](executor)Create workload thread pool without cgroup #31170
2024-02-21 13:53:18 +08:00
1d6dc9a5f0
[fix](catalog recycle bin) Forbid recover partition if table schame is changed #31146
2024-02-21 13:53:18 +08:00
872e1f6687
[fix](backup) fix concurrent upload and release snapshot crash ( #31144 )
...
In upload implementation, filesystem::size throws an exception
if the specified file, which is removed by the release snapshot
task, does not exist.
2024-02-21 13:53:18 +08:00
27ae96c5d0
[fix](case) adjust case if running on muti BEs ( #31160 )
...
Co-authored-by: stephen <hello-stephen@qq.com >
2024-02-21 13:53:18 +08:00
e2514c950c
Fix load stats cache FE ut case. ( #31162 )
2024-02-21 13:53:18 +08:00
7c1bb4cb22
[fix](docs) add joinHint.md to sidebars ( #31155 )
2024-02-21 13:53:18 +08:00
87b5ed187e
Fix hive p2 case ( #31149 )
2024-02-21 13:53:18 +08:00
99cdbc7e12
[fix](memory) Fix jemalloc lib name for thirdparty arrow ( #28843 )
...
after #28429
2024-02-21 13:52:10 +08:00
0ab8143565
[fix](catalog recycle bin) table partition meta is error if recover partition in some case ( #31125 )
2024-02-21 13:52:10 +08:00
4735c5b50f
2.1.0-rc08
2024-02-20 23:45:35 +08:00
2af920c6f8
[chore](regression) Update q03.sql ( #31063 )
...
add 'order by' of case
2024-02-20 16:24:05 +08:00
97c9d75af3
[Feature](executor)Add scan_thread_num property for workload group ( #31106 )
2024-02-20 16:24:05 +08:00
6d7662e8ac
[fix](tools) Fix non-standard code in tpch/tpcds dbgen tools leading to compilation failure ( #31124 )
...
Fix non-standard code in tpch/tpcds dbgen tools leading to compilation failure
2024-02-20 16:24:05 +08:00
9251816673
[fix](Nereids): log constraint editlog in lock scope ( #30630 )
2024-02-20 16:24:05 +08:00
95a1e1c0bf
[Enhancement](tools) Support transaction for benchmarks loading ( #31126 )
2024-02-20 16:24:05 +08:00
4aaab6fb44
[fix](statistics)Refresh follower FE cache after alter column stats. Support alter index column stats ( #31108 )
...
1. Refresh follower FE cache after alter column stats. So that follower could update the cached stats too.
2. Support alter index column stats.
2024-02-20 16:23:53 +08:00
54e8354fc6
[doc](mtmv) Update the mtmv documentation to make it consistent with the code ( #30899 )
2024-02-20 16:23:53 +08:00
493385c2c7
[Bug](AggState) fix not match function when agg combinator function has alias ( #31101 )
2024-02-20 16:23:53 +08:00
fd453ace38
[enhancement](sc-test) Optimize waitForSchemeChangeDone ( #31002 )
2024-02-20 16:23:53 +08:00
a7a530189e
[regression-test](decimalv3) Adding numerical precision exceeding the data type will error case ( #31068 )
...
* Adding numerical precision exceeding the data type will result in an error case
* add order by
---------
Co-authored-by: smallhibiscus <844981280>
2024-02-20 16:23:53 +08:00
15964ee3d8
[opt](deps) reduce the size of FE dependencies ( #31129 )
2024-02-20 16:23:53 +08:00
b69acbf98b
[Improvement](group commit) FE should throw NotFoundException if database or table not exist when loading data ( #31113 )
...
* [Improvement](group commit) FE should throw NotFoundException if database or table not exist when loading data
* edit
2024-02-20 16:23:53 +08:00
24fb24db3e
[refactor](catalog) catalog interface refactor #31047 ( #31135 )
2024-02-20 11:08:50 +08:00
6778b4ed93
[pipelineX](refactor) make non-virtual function in Dependency ( #31109 )
...
* [pipelineX](refactor) make non-virtual function in Dependency
* update
2024-02-20 09:18:33 +08:00
76767f2867
[fix](fe) Fix UnsetVariableStmt write editlog in non master node ( #31080 )
...
* Problem introduced by https://github.com/apache/doris/pull/27552
2024-02-20 09:18:33 +08:00
689b2c7bc5
[fix](s3client) fix unable to access COS ( #31111 )
2024-02-20 09:16:28 +08:00
02bded2688
[Improve](common)Optimize logging performance with LOG.isDebugEnabled() ( #31091 )
...
* [Improve](common)Optimize logging performance with LOG.isDebugEnabled()
* fix error ut
2024-02-20 09:16:14 +08:00
4c34ebb1cf
[fix](mtmv)Fix the case failure issue caused by the same catalog name #31058
2024-02-20 09:12:38 +08:00
faaa51fa9d
[fix](regression-test) test_nested_type_with_rowstore is flaky ( #31121 )
2024-02-20 09:12:38 +08:00
7ca3be6d51
[fix](parquet) return error if schema changed in complex types ( #31128 )
...
Check the column type of complex type to prevent core dump in BE. ColumnReader will throw segmentation fault in the following case:
Change complex types in hive:
hive> create table struct_test(
id int,
sf struct<f1: int, f2: map<string, string>>) stored as parquet;
hive> insert into struct_test values
(1, named_struct('f1', 1, 'f2', str_to_map('1:s2,2:s2'))),
(2, named_struct('f1', 2, 'f2', str_to_map('k1:s3,k2:s4'))),
(3, named_struct('f1', 3, 'f2', str_to_map('k1:s5,k2:s6')));
hive> alter table struct_test change sf sf struct<f1:int, f2: string>;
2024-02-20 09:12:38 +08:00
8f70c00a26
[fix](routine-load) add lock to avoid editlog out of order when concurrent update job ( #31095 )
2024-02-20 09:12:38 +08:00
7a1bd6abb0
[improvment](group_commit) Refector scan wal function ( #30939 )
...
Co-authored-by: Yongqiang YANG <98214048+dataroaring@users.noreply.github.com >
2024-02-20 09:12:38 +08:00
066d674358
[Fix](inverted index) fix inverted index read data opt not work on MOW ( #31075 )
2024-02-20 09:12:38 +08:00
4a33d9820a
[fix](multi-catalog)fix getting ugi methods and unify them ( #30844 )
...
put all ugi login methods to HadoopUGI
2024-02-20 09:12:38 +08:00
d55d796c74
[branch-2.1](multi-table-load) reduce routine load job transaction info storage in memory #31127
2024-02-20 09:08:56 +08:00
9a708806e0
[fix](segcompaction) enable segcompaction by default ( #30810 )
2024-02-19 19:04:22 +08:00
56ebff8f74
Doris doc ( #31099 )
2024-02-19 18:35:29 +08:00
e892b967da
[docs](functions) fix and complete agg functions docs #31114
2024-02-19 18:35:29 +08:00
