842a5b8e24
[refactor](agg) Abstract the hash operation into a method" ( #11399 )
2022-08-02 17:27:19 +08:00
1db8a2d136
[bugfix](runtimefilter)fix runtimefilter access violation when stub is nullptr ( #11180 )
2022-08-02 16:57:17 +08:00
945256f6a6
[regression](Nereids) add nereids syntax regression suite ( #11423 )
...
add a new regression suite for test nereids syntax. It load data only and has no test case now.
2022-08-02 16:45:53 +08:00
ff8a4ec2f2
[WIP][feature] support create,alter,refresh,drop stmt syntax for multi table materialized view ( #11218 )
...
* [WIP][feature] support `create`,`alter`,`refresh`,`drop` stmt for multi
table materialized view
2022-08-02 16:03:36 +08:00
cd6fbd09bf
[feature] (Nereids) add rule to merge consecutive project nodes ( #11262 )
...
Merge consecutive project nodes. For example:
logical plan tree:
```
project(a)
|
project(a,b)
|
project(a, b, c)
|
scan
```
transformed to:
```
project(a)
|
scan
```
2022-08-02 15:58:55 +08:00
38ffe685b5
[Bug](ODBC) fix vectorized null value error report in odbc scan node ( #11420 )
...
* [Bug](ODBC) fix vectorized null value error report in odbc scan node
Co-authored-by: lihaopeng <lihaopeng@baidu.com >
2022-08-02 15:44:12 +08:00
e41bfd8149
[Bug] Update Stmt Fail use in predicate query ( #11414 )
...
Co-authored-by: lihaopeng <lihaopeng@baidu.com >
2022-08-02 15:14:36 +08:00
fcfc76f4a9
[Bug](date function) from_unixtime return wrong result ( #11410 )
...
* [Bug](date function) `from_unixtime` return wrong result
2022-08-02 14:23:54 +08:00
ac62c9507e
[improvement](script)Audit build script ( #11411 )
2022-08-02 12:06:44 +08:00
f730a048b1
[feature-wip](load) Support single replica load ( #10298 )
...
During load process, the same operation are performed on all replicas such as sort and aggregation,
which are resource-intensive.
Concurrent data load would consume much CPU and memory resources.
It's better to perform write process (writing data into MemTable and then data flush) on single replica
and synchronize data files to other replicas before transaction finished.
2022-08-02 11:44:18 +08:00
abbf75d302
[doc][refactor](metrics) Reorganize FE and BE metrics and add document ( #11307 )
2022-08-02 11:34:06 +08:00
60b5ed16a8
[improvement](test) move correctness and account suites to p0 while tpcds_sf1 t0 p1 ( #11350 )
2022-08-02 11:23:01 +08:00
8e3979b3d1
FIX: audit user ( #11408 )
2022-08-02 11:19:02 +08:00
f8f63874a3
[docs](routine-load)modify config name ( #11394 )
...
modify config name
2022-08-02 11:17:37 +08:00
8537ec6c02
[doc] Add missing ADMIN DIAGNOSE TABLET doc ( #11273 )
...
Add missing ADMIN DIAGNOSE TABLET doc
2022-08-02 11:13:57 +08:00
cee4e767b5
fix document ( #11383 )
...
fix document : en/get-starting
2022-08-02 11:12:57 +08:00
22f5dedf8a
[doc]Doc fix ( #11384 )
...
* fix document，pull-request and get-starting
2022-08-02 11:12:11 +08:00
0ac5228c05
[feature-wip][multi-catalog]Support prefetch for orc file format ( #11292 )
...
Refactor the prefetch code in parquet and support prefetch for orc file format
2022-08-02 11:01:15 +08:00
bd6e3cf132
[improvement]lock_times_limit ( #11404 )
...
Co-authored-by: songning03 <songning03@meituan.com >
2022-08-02 10:59:58 +08:00
44a1a20e65
[feature-wip](parquet-reader)parse parquet schema ( #11381 )
...
Analyze schema elements in parquet FileMetaData, and generate the hierarchy of nested fields.
For exmpale:
1. primitive type
```
// thrift:
optional int32 <column-name>;
// sql definition:
<column-name> int32;
```
2. nested type
```
// thrift:
optional group <column-name> (LIST) {
repeated group bag {
optional group array_element (LIST) {
repeated group bag {
optional int32 array_element
}
}
}
}
// sql definition:
<column-name> array<array<int32>>
```
2022-08-02 10:56:13 +08:00
003335c1c5
[refactor](schema change) spark dpp need not call convert rowset during load process ( #11397 )
...
* remove unused schema change logic in push handler
Co-authored-by: yiguolei <yiguolei@gmail.com >
2022-08-02 10:18:00 +08:00
6af35a8eb7
[fix](TabletInvertedIndex) fix potential deadlock between ForkJoinPool and TabletInvertedIndex ( #11365 )
...
* [fix](TabletInvertedIndex) fix potential deadlock between ForkJoinPool and TabletInvertedIndex
The default ForkJoinPool is shared by all parallelStream by default, and we obtain read lock outside the ForkJoinPool in TabletInvertIndex while we obtain read lock inside the same ForkJoinPool in TabletStatMgr which may cause deadlock
2022-08-02 10:08:05 +08:00
5c1cd058f2
[Feature] Add interface to check tablet segment lost ( #10711 )
...
Co-authored-by: weizuo <weizuo@xiaomi.com >
2022-08-02 09:40:04 +08:00
edf92f10ff
[fix](build) The nproc command does not exist in macos by default, use sysctl -n hw.logicalcpu instead to get the number of logical cores available ( #11379 )
2022-08-02 09:11:14 +08:00
27a19118dd
[Community](about)Update .asf.yaml ( #11390 )
2022-08-01 23:15:37 +08:00
27be5e8667
[feature-wip](decimalv3) Fix UTs when decimalv3 is enabled ( #11380 )
2022-08-01 23:07:38 +08:00
dddacd120d
[fix](planner) Fix inconsistent nullability between outputTuple and groupByExpr when executing agg query ( #11361 )
2022-08-01 22:57:51 +08:00
80ce027ea2
[refactor](Nereids): refactor UT of Nereids ( #11330 )
...
refactor the UT of Nereids.
Extract the plan constructor (This PR extract all olapscan and table into PlanConstructor).
2022-08-01 22:53:00 +08:00
4ccdd65bf6
[Fix](array) fix mysql_row_buffer may use after free when reserve() delete original address in dynamic_mode ( #11395 )
...
```
if (!_dynamic_mode) {
int8store(_len_pos, _pos - _len_pos - 8);
_len_pos = nullptr;
}
```
_len_pos may be pointed to the pos which already deleted in reserve, int8store will asign value to the freed address,
and lead to use after free when build in ASAN.So I changed _len_pos to the offset of _buf
2022-08-01 22:52:19 +08:00
667689e9ba
[Fix](array) fix array permute ( #11389 )
2022-08-01 22:46:03 +08:00
69961ac19c
fix bug for lower case when rename table name ( #11403 )
2022-08-01 22:30:41 +08:00
6da6ffb836
[feature] (Nereids) add rewrite rule to merge consecutive filter nodes ( #11248 )
...
this rule aims to merge consecutive filters.
For example:
logical plan tree:
```
project
|
filter(a>0)
|
filter(b>0)
|
scan
```
transformed to:
```
project
|
filter(a>0 and b>0)
|
scan
```
2022-08-01 17:29:15 +08:00
69bfbae856
[enhancement](nereids) Normalize expressions before performing plan rewriting ( #11299 )
...
Rules for normalizing expressions should be applied once before do some extra expression transforms.
Normalization rules include:
1. NormalizeBinaryPredicatesRule
2. BetweenToCompoundRule
3. SimplifyNotExprRule
2022-08-01 17:15:04 +08:00
eb778da64a
Add ssb sf0.1 test under unique table with zstd ( #11348 )
...
Co-authored-by: smallhibiscus <844981280>
2022-08-01 17:04:55 +08:00
7ca6f8b6a2
[refactor](Nereids)split rewrite and insert into memo to 2 functions ( #11342 )
...
Split rewrite and insert into memo to 2 functions to make the code easy to read.
2022-08-01 14:44:01 +08:00
8287115ec3
[test]Delete some unnecessary information ( #11382 )
2022-08-01 14:31:39 +08:00
8f85ad1c96
[doc](intro) update the introduce ( #11364 )
2022-08-01 14:22:17 +08:00
8a8e8e8b45
[fix](stop-script) use kill -9 to stop fe as usual ( #11387 )
2022-08-01 14:18:10 +08:00
1cf57a985d
[fix] Fix the query result error caused by the grouping sets statemen… ( #11316 )
...
* [fix] Fix the query result error caused by the grouping sets statement grouping as an expression
2022-08-01 13:52:18 +08:00
4f5e1601df
[bug](scanner) Improve limit query performance on olapScannode and avoid infinite loop ( #11301 )
...
1. Fix a bug that query large column table may cause infinite loop
2. Optimize the query logic with limit, for the case where the limit value is relatively small, reduce the parallelism of the scanner, reduce unnecessary resource consumption, and increase the number of similar queries that the system can carry at the same time, and increase the query speed by more than 60%
2022-08-01 13:50:12 +08:00
b35daf0a04
[improvement](light-schema-change) Support tablet schema cache ( #11131 )
2022-08-01 12:18:00 +08:00
0325fa436e
[fix](agg)Add field of 'is_first_phase' in TAggregationNode ( #11321 )
2022-08-01 11:49:50 +08:00
52460af74b
[Bug][Vectorized] Support the .* in hyperscan to valid the % in SQL ( #11371 )
...
Co-authored-by: lihaopeng <lihaopeng@baidu.com >
2022-08-01 11:00:05 +08:00
34ed40bb71
[Feature](array) doe support array ( #11340 )
2022-08-01 10:56:40 +08:00
76970200ec
[Bugfix](light-schema-change) Fix bf columns message miss when load ( #11367 )
2022-08-01 10:53:25 +08:00
11f6c69a22
[bugfix](start script)fix a bug while JAVA_HOME not set ( #11366 )
2022-08-01 10:44:03 +08:00
547c18d28b
[feature](load) support CLEAN LABEL stmt ( #11362 )
2022-08-01 10:43:33 +08:00
d360974dce
[improvement](agg)Use phmap::flat_hash_set in AggregateFunctionUniq ( #11363 )
...
This reverts commit 688b55053dd1fc5113343a6f565ad732ddd9612a.
2022-08-01 10:36:11 +08:00
61b157abe3
[bug](storage-scanner) fix infinite loop when reading wide table ( #11372 )
...
* fix infinite loop when reading wide table
When a wide table is read, the 1st batch would be exceed raw_bytes_threshold,
so Scanner should read at least 1 row.
Actually, we should adjust batch size automatically to reduce memoery usage.
2022-08-01 10:33:04 +08:00
3f43d7262b
[Bug] (datetimev2) fix bugs for datev2/datetimev2 ( #11358 )
2022-08-01 10:29:56 +08:00
