2cd3bf80dc
[bugfix](schema change)fix core dump on vectorized_alter_table ( #11538 )
2022-08-08 10:45:28 +08:00
1e6a3610a7
[feature-wip](unique-key-merge-on-write) optimize rowid conversion and add ut ( #11541 )
2022-08-08 10:41:44 +08:00
e8a344b683
[feature-wip](parquet-reader) add predicate filter and column reader ( #11488 )
2022-08-08 10:21:24 +08:00
bd4048f8fb
[enhancement](compaction) add idle schedule and max_size limit for base compaction ( #11542 )
...
Co-authored-by: yixiutt <yixiu@selectdb.com >
2022-08-07 16:21:57 +08:00
95753ec868
[feature](parquet-reader) add group filter util ( #11533 )
...
* [feature-wip](parquet-reader) add group filter util
Co-authored-by: jinzhe <jinzhe@selectdb.com >
2022-08-05 14:02:48 +08:00
321107cb40
[refactor](schema change) Using tablet schema shared ptr instead of raw ptr ( #11475 )
...
* Using tabletschema shared ptr instead of raw ptrs
Co-authored-by: yiguolei <yiguolei@gmail.com >
2022-08-05 11:04:38 +08:00
6eb8ac0ebf
[feature-wip][multi-catalog]Support caseSensitive field name in file scan node ( #11310 )
...
* Impl case sentive in file scan node
2022-08-05 08:03:16 +08:00
b5531c5caf
[BugFix](BE) fix condition index doesn't match ( #11474 )
...
* [BugFix](Be) fix condition index doesn't match
2022-08-05 07:57:18 +08:00
092a394782
[improvement](agg)limit the output of agg node ( #11461 )
...
* [improvement](agg)limit the output of agg node
2022-08-05 07:53:55 +08:00
75fc830573
[Bug](date function) fix wrong year for format '%x' ( #11520 )
2022-08-05 06:22:22 +08:00
b6118acc19
[feature](functions) support add_months on vectorized engine ( #11518 )
2022-08-04 21:39:10 +08:00
346fdeeee0
[fix](ut) Fix BE UT BetaRowsetTest failed ( #11500 )
2022-08-04 17:57:57 +08:00
aed0282046
[feature-wip](parquet-reader)get compressed parquet page data ( #11493 )
2022-08-04 17:44:52 +08:00
ec3c911f97
[Feature][Materialized-View] support materialized view on vectorized engine ( #10792 )
2022-08-04 14:07:48 +08:00
a943adac1a
[feature](cache) Add FileCache for RemoteFile ( #11186 )
...
Add FileCache for RemoteFile, it will be opened in StoragePolicy.
Cold data in remote file will be download to local cache files.
2022-08-04 10:57:32 +08:00
ecbf87d77b
[bugfix](memtracker)fix exceed memory limit log ( #11485 )
2022-08-04 10:22:20 +08:00
ce68d24e95
[Bug](function) fix current_date not equal to curdate ( #11463 )
...
* fix current_date not equal to curdate
2022-08-04 09:25:50 +08:00
838fdc1354
[Bug](httpserver) Fix bug that http server should not be stoped in destructor if it not running
...
Co-authored-by: weizuo <weizuo@xiaomi.com >
2022-08-03 19:44:46 +08:00
e1b878fe10
[Improvement](datev2) apply time LUT to datev2/datetimev2 ( #11401 )
...
* [Improvement](datev2) apply time LUT to datev2/datetimev2
2022-08-03 17:15:09 +08:00
5b9b6c9065
[WIP](decimalv3) WIP ( #11443 )
...
* [feature-WIP](decimalv3) fix some bugs of decimalv3
2022-08-03 11:21:36 +08:00
77d82bb292
[Bug](MaterializedView) Fix bug of light schema change do not set right unique id cause MV coredump ( #11396 )
...
Fix bug of light schema change do not set right unique id cause MV coredump
2022-08-03 11:21:28 +08:00
b892dfdbbd
[Improvement](regresstion test) Fix regression test case failure for ASAN build ( #11400 )
...
* [improvement](regresstion test) Improve performance of ASAN build by using -O3 and fix mem limit exceed error for nereids test cases
* exclude tpcds_sf1 q72 for ASAN build because this query takes too long time
2022-08-03 11:19:50 +08:00
573ebf235e
[enhancement](build) Support customizing extra compile flags ( #11444 )
2022-08-03 11:02:17 +08:00
1b4d6a620a
(feature-wip)[parquet-reader] support page index serde ( #11415 )
2022-08-03 10:36:06 +08:00
de4466624d
[refactor](schema change)Remove delete from sc ( #11441 )
...
* not need call delete handler to filter rows since they are filtered in rowset reader
* need not call delete eval in schema change and remove related code
Co-authored-by: yiguolei <yiguolei@gmail.com >
2022-08-03 03:29:41 +08:00
842a5b8e24
[refactor](agg) Abstract the hash operation into a method" ( #11399 )
2022-08-02 17:27:19 +08:00
1db8a2d136
[bugfix](runtimefilter)fix runtimefilter access violation when stub is nullptr ( #11180 )
2022-08-02 16:57:17 +08:00
38ffe685b5
[Bug](ODBC) fix vectorized null value error report in odbc scan node ( #11420 )
...
* [Bug](ODBC) fix vectorized null value error report in odbc scan node
Co-authored-by: lihaopeng <lihaopeng@baidu.com >
2022-08-02 15:44:12 +08:00
f730a048b1
[feature-wip](load) Support single replica load ( #10298 )
...
During load process, the same operation are performed on all replicas such as sort and aggregation,
which are resource-intensive.
Concurrent data load would consume much CPU and memory resources.
It's better to perform write process (writing data into MemTable and then data flush) on single replica
and synchronize data files to other replicas before transaction finished.
2022-08-02 11:44:18 +08:00
abbf75d302
[doc][refactor](metrics) Reorganize FE and BE metrics and add document ( #11307 )
2022-08-02 11:34:06 +08:00
0ac5228c05
[feature-wip][multi-catalog]Support prefetch for orc file format ( #11292 )
...
Refactor the prefetch code in parquet and support prefetch for orc file format
2022-08-02 11:01:15 +08:00
bd6e3cf132
[improvement]lock_times_limit ( #11404 )
...
Co-authored-by: songning03 <songning03@meituan.com >
2022-08-02 10:59:58 +08:00
44a1a20e65
[feature-wip](parquet-reader)parse parquet schema ( #11381 )
...
Analyze schema elements in parquet FileMetaData, and generate the hierarchy of nested fields.
For exmpale:
1. primitive type
```
// thrift:
optional int32 <column-name>;
// sql definition:
<column-name> int32;
```
2. nested type
```
// thrift:
optional group <column-name> (LIST) {
repeated group bag {
optional group array_element (LIST) {
repeated group bag {
optional int32 array_element
}
}
}
}
// sql definition:
<column-name> array<array<int32>>
```
2022-08-02 10:56:13 +08:00
003335c1c5
[refactor](schema change) spark dpp need not call convert rowset during load process ( #11397 )
...
* remove unused schema change logic in push handler
Co-authored-by: yiguolei <yiguolei@gmail.com >
2022-08-02 10:18:00 +08:00
5c1cd058f2
[Feature] Add interface to check tablet segment lost ( #10711 )
...
Co-authored-by: weizuo <weizuo@xiaomi.com >
2022-08-02 09:40:04 +08:00
27be5e8667
[feature-wip](decimalv3) Fix UTs when decimalv3 is enabled ( #11380 )
2022-08-01 23:07:38 +08:00
4ccdd65bf6
[Fix](array) fix mysql_row_buffer may use after free when reserve() delete original address in dynamic_mode ( #11395 )
...
```
if (!_dynamic_mode) {
int8store(_len_pos, _pos - _len_pos - 8);
_len_pos = nullptr;
}
```
_len_pos may be pointed to the pos which already deleted in reserve, int8store will asign value to the freed address,
and lead to use after free when build in ASAN.So I changed _len_pos to the offset of _buf
2022-08-01 22:52:19 +08:00
667689e9ba
[Fix](array) fix array permute ( #11389 )
2022-08-01 22:46:03 +08:00
1cf57a985d
[fix] Fix the query result error caused by the grouping sets statemen… ( #11316 )
...
* [fix] Fix the query result error caused by the grouping sets statement grouping as an expression
2022-08-01 13:52:18 +08:00
4f5e1601df
[bug](scanner) Improve limit query performance on olapScannode and avoid infinite loop ( #11301 )
...
1. Fix a bug that query large column table may cause infinite loop
2. Optimize the query logic with limit, for the case where the limit value is relatively small, reduce the parallelism of the scanner, reduce unnecessary resource consumption, and increase the number of similar queries that the system can carry at the same time, and increase the query speed by more than 60%
2022-08-01 13:50:12 +08:00
b35daf0a04
[improvement](light-schema-change) Support tablet schema cache ( #11131 )
2022-08-01 12:18:00 +08:00
0325fa436e
[fix](agg)Add field of 'is_first_phase' in TAggregationNode ( #11321 )
2022-08-01 11:49:50 +08:00
52460af74b
[Bug][Vectorized] Support the .* in hyperscan to valid the % in SQL ( #11371 )
...
Co-authored-by: lihaopeng <lihaopeng@baidu.com >
2022-08-01 11:00:05 +08:00
d360974dce
[improvement](agg)Use phmap::flat_hash_set in AggregateFunctionUniq ( #11363 )
...
This reverts commit 688b55053dd1fc5113343a6f565ad732ddd9612a.
2022-08-01 10:36:11 +08:00
61b157abe3
[bug](storage-scanner) fix infinite loop when reading wide table ( #11372 )
...
* fix infinite loop when reading wide table
When a wide table is read, the 1st batch would be exceed raw_bytes_threshold,
so Scanner should read at least 1 row.
Actually, we should adjust batch size automatically to reduce memoery usage.
2022-08-01 10:33:04 +08:00
3f43d7262b
[Bug] (datetimev2) fix bugs for datev2/datetimev2 ( #11358 )
2022-08-01 10:29:56 +08:00
73d8f5901d
fix mem tracker limiter ( #11376 )
2022-08-01 09:44:04 +08:00
688b55053d
Revert "[improvement]Use phmap::flat_hash_set in AggregateFunctionUniq ( #11257 )" ( #11356 )
...
This reverts commit a7199fb98e18b925664b38460b667d04cbee8e01.
2022-07-30 23:15:36 +08:00
2783267599
[feature-wip](unique-key-merge-on-write) update bitmap after compaction, DSIP-018 ( #11289 )
2022-07-30 19:50:47 +08:00
9333e79ae0
[feature-wip](unique-key-merge-on-write) Add support for tablet migration, DSIP-018[5/3] ( #11283 )
2022-07-30 19:50:11 +08:00
