1fd637eab7
[Docs](Job)Add Job and Job tasks docs ( #23576 )
2023-08-29 10:06:49 +08:00
da9eb79ac4
[Enhancement](Schema hash) Remove schema hash in tablet info ( #23516 )
2023-08-29 10:05:12 +08:00
d863cc3a12
[fix](move-memtable) fix tablets to commit ( #23577 )
2023-08-29 09:49:07 +08:00
b2cb0d99d4
[opt](docs) opt doc to avoid users using port 8030 everywhere ( #23379 )
2023-08-29 09:47:24 +08:00
9c65b7ab96
[improvement](column_reader) move load once to index reader to reduce ( #23537 )
...
memory footprint of column reader
2023-08-29 09:34:27 +08:00
fbf8499999
[improvement](compaction) reduce the memory using on vertical compaction ( #23388 )
2023-08-28 21:54:21 +08:00
6f3e2a30e6
[Feat](Nereids) Add leading and ordered hint ( #22057 )
...
Add leading hint and ordered hint. Usage:
select /*+ ordered / * from a join b on xxx; which will limit join order to original order
select /+ leading ({b a}) */ from a join b on xxx; which will change join order to b join a.
2023-08-28 21:04:40 +08:00
21fefb2831
[improvement](build) Compile enable WITH_MYSQL ( #23371 )
2023-08-28 20:41:06 +08:00
35a1404bbe
[fix](load) add error handle when load data dir ( #23457 )
2023-08-28 19:33:50 +08:00
0ff191cdf3
[feature](Nereids) add expr depth limit and expr children limit in Nereids ( #23569 )
...
#### `expr_depth_limit`
Default：3000
IsMutable：true
Limit on the depth of an expr tree. Exceed this limit may cause long analysis time while holding db read lock. Do not set this if you know what you are doing
#### `expr_children_limit`
Default：10000
IsMutable：true
Limit on the number of expr children of an expr tree. Exceed this limit may cause long analysis time while holding database read lock.
2023-08-28 19:03:43 +08:00
a70ebe87c5
[optimize](Nereids): speedup analyze ( #23549 )
...
- avoid some `withRowCount`
- ArrayList with size
- checkPrimitiveInputDataTypesWithExpectType avoid to check AnyDataType
2023-08-28 18:17:55 +08:00
8e4c0d1e81
[Bug](materialized-view) fix divide double can not match mv ( #23504 )
...
* fix divide double can not match mv
* fix
* fix
2023-08-28 18:01:08 +08:00
392437008c
[Improvement](ColumnReader) optimize memory using of ColumnReader meta ( #23528 )
2023-08-28 17:57:59 +08:00
7e70c61766
[regression](nereids) add test case for runtime filter check on tpcds_sf100/tpch_sf1000 ( #23451 )
...
* add tpch/ds runtime filter check
2023-08-28 17:34:35 +08:00
10792ca0f7
[fix](nereids) Mistaken stats when analyzing table incrementally and partition number less than 512 #23507
...
Fix bug that mistaken stats when analyzing table incrementally and partition number less than 512
Fix bug that cron expression lost during analyzing
Mark system job as running after registered to AnalysisManager to avoid submit same jobs if previous one take long time
2023-08-28 17:31:36 +08:00
1cc6474487
[fix](planner)fix bug of pushing conjunct through agg node ( #23483 )
2023-08-28 17:10:13 +08:00
2dff89a77a
[Log](Alter) Print table's state when checkNormalStateForAlter() failed ( #23358 )
2023-08-28 17:04:01 +08:00
650cc25ea4
[fix](light-schema-change) fix schema consistency check failed ( #23283 )
2023-08-28 16:40:30 +08:00
6ac694aede
[Configuration](multi-catalog) Modify default external cache expire time to 10 mins. ( #23490 )
...
Configuration Modify default external cache expire time to 10 mins.
2023-08-28 16:16:43 +08:00
29b94c4ed7
[pipeline](refactor) refine pipeline fragment context ( #23478 )
2023-08-28 15:55:02 +08:00
7e7cfd17bf
[fix](tablet sink) check data valid of tablet sink data ( #23530 )
...
Co-authored-by: Yongqiang YANG <98214048+dataroaring@users.noreply.github.com >
2023-08-28 15:54:12 +08:00
f70638e895
[Fix](autobucket) Fix autobucket partition size by using getAllDataSize including cooldown size ( #23557 )
2023-08-28 15:24:48 +08:00
6e82178847
[Bug](materialized-view) fix loaddb analyze failed on MaterializedIndexMeta ( #23442 )
...
* fix loaddb analyze failed on MaterializedIndexMeta
* update
* update
2023-08-28 15:18:18 +08:00
3049533e63
[Bug](materialized-view) fix core dump on create materialized view when diffrent mv column have same reference base column ( #23425 )
...
* Remove redundant predicates on scan node
update
fix core dump on create materialized view when diffrent mv column have same reference base column
Revert "update"
This reverts commit d9ef8dca123b281dc8f1c936ae5130267dff2964.
Revert "Remove redundant predicates on scan node"
This reverts commit f24931758163f59bfc47ee10509634ca97358676.
* update
* fix
* update
* update
2023-08-28 14:40:51 +08:00
bea5701cce
[minor](docs) update docs for variable sql_select_limit ( #23262 )
2023-08-28 14:38:32 +08:00
28a2e71084
[pipelineX](refactor) refine codes ( #23521 )
...
* [pipelineX](refactor) refine codes
* update
* update
2023-08-28 14:38:07 +08:00
4c8fc06e40
[Feature](fe) Add admin set partition version statement ( #23086 )
...
This commit add a statement to modify partition visible version.
2023-08-28 14:31:54 +08:00
82fe5aa5a0
[fix](regression) rename tables in test_stream_load_move_memtable ( #23545 )
2023-08-28 14:31:00 +08:00
83467e5d9e
[docs](website) fix a typo in docs tittle ( #23431 )
2023-08-28 14:27:45 +08:00
c05319b8eb
[fix](agg) incorrect result of bitmap_agg and bitmap_union ( #23558 )
2023-08-28 14:22:19 +08:00
f7d2c1faf6
[feature](Nereids) support select key encryptKey ( #23257 )
...
Add select key
```
- CREATE ENCRYPTKEY key_name AS "key_string"
- select key my_key
+-----------------------------+
| encryptKeyRef('', 'my_key') |
+-----------------------------+
| ABCD123456789 |
+-----------------------------+
```
2023-08-28 14:07:26 +08:00
5be8d57f52
[fix](be-ut) fix ColumnFixedLenghtObjectTest on 32 bits system ( #23519 )
2023-08-28 14:02:05 +08:00
ef2fc44e5c
[Improve](Job)Allows modify the configuration of the Job queue and the number of consumer threads ( #23547 )
2023-08-28 12:01:49 +08:00
2799393a50
forbid: test_stream_load_move_memtable ( #23556 )
2023-08-28 11:41:08 +08:00
e84989fb6d
[feature](Nereids) support map type ( #23493 )
2023-08-28 11:31:44 +08:00
b181a9f099
[feature](Nereids) support array type in fold constant framework ( #23373 )
...
1. use legacy planner way to process constant folding result from be
2. support signature with complex type for constant folding on fe
2023-08-28 10:47:43 +08:00
962221cb18
[test](log) add log for debug case failure ( #23506 )
2023-08-28 10:45:25 +08:00
d19dcd6bc1
[improve](jdbc catalog) support sqlserver uniqueidentifier data type ( #23297 )
2023-08-28 10:30:10 +08:00
eadffedb33
[Feature](fe) Add admin set table status statement ( #23139 )
...
For some certain bugs, jobs is stuck in FE by the table state. For example, There is a bug which causes table remains ROLLUP state after adding rollup job, then other alter jobs later will not succeed because the table state is always ROLLUP but not NORMAL.
This commit adds a statement which is used to set the state of the specified table.
2023-08-28 10:22:09 +08:00
4dbec854e4
[Docs](inverted index) add tokenize function doc ( #23518 )
2023-08-28 10:19:03 +08:00
92bdf75836
[fix](Nereids): LogicalRepeat equals lack @Override ( #23408 )
2023-08-28 10:07:39 +08:00
981586155c
[Improvement][json] optimize performance of json_extract by reusing json path object ( #23430 )
...
* reuse json path to speed up json function
* fix typo
* clang format
* path reentry safe
* fix compile error
* fix bug of continue
2023-08-27 17:39:10 +08:00
e0bf621fe0
[chore](build) Fix compilation errors for BE UT ( #23535 )
...
Issue Number: close #23536
This issue was introduced by #23414 .
2023-08-27 11:52:13 +08:00
153e8f0f72
[imporvement](table property) support for alter table property: skip wirte index , single compaction ( #23475 )
2023-08-26 23:52:09 +08:00
ba351af452
[enhancement](thirdparty) upgrade thirdparty libs - again ( #23414 )
...
submit again #23290 (not upgrade brpc, because bthread local has error)
protobuf 3.15.0 -> 21.11
glog 0.4.0 -> 0.6.0
lz4 1.9.3 -> 1.9.4
curl 7.79.0 -> 8.2.1
zstd 1.5.2 -> 1.5.5
arrow 7.0.0 -> 13.0.0
abseil 20220623.1 -> 20230125.3
orc 1.7.2 -> 1.9.0
jemalloc for arrow 5.2.1 -> 5.3.0
xsimd 7.0.0 -> 13.0.0
opentelemetry-proto 0.19.0 -> 1.0.0
opentelemetry 1.8.3 -> 1.10.0
new:
c-ares -> 1.19.1
grpc -> 1.54.3
2023-08-26 22:59:10 +08:00
93918253ba
[fix](metric) fix issue that the counter of rejected transactions does not cover some failed situations for load ( #23363 )
...
Co-authored-by: caiconghui1 <caiconghui1@jd.com >
2023-08-26 20:06:42 +08:00
a5761a25c5
[feature](move-memtable)[7/7] add regression tests ( #23515 )
...
Co-authored-by: laihui <1353307710@qq.com >
2023-08-26 17:52:10 +08:00
30658ebeda
[Fix](planner) Fix query queue can not limit maxConcurrency #23418
...
2 Fix concurrent can not limit
2023-08-26 17:31:44 +08:00
30e3c5bbe6
[bugfix](file cache) Fix the init file cache coredump ( #23464 )
...
* [bugfix](file cache) Fix the init file cache coredump
* fix compile
2023-08-26 16:50:50 +08:00
40be6a0b05
[fix](hive) do not split compress data file and support lz4/snappy block codec ( #23245 )
...
1. do not split compress data file
Some data file in hive is compressed with gzip, deflate, etc.
These kinds of file can not be splitted.
2. Support lz4 block codec
for hive scan node, use lz4 block codec instead of lz4 frame codec
4. Support snappy block codec
For hadoop snappy
5. Optimize the `count(*)` query of csv file
For query like `select count(*) from tbl`, only need to split the line, no need to split the column.
Need to pick to branch-2.0 after this PR: #22304
2023-08-26 12:59:05 +08:00
