zhengyu 62ec74f4e7 segcompaction featuring verticalcompaction (#16731) ...

This patchset applies the following changes:

using vertical compaction machanism to do segcompaction
basic (WIP) refraction to separate segcompaction logic from BetaRowsetWriter
add segcompaction specific ut and regression tests

2023-03-01 10:55:40 +08:00

..

account_p0

[feature](auth)Implementing privilege management with rbac model (#16091)

2023-02-10 12:30:49 +08:00

bitmap_functions

[fix](planner) Keep type of null literal expr when register conjuncts (#15878)

2023-01-17 16:48:02 +08:00

bloom_filter_p0

[fix](nereids)translate is not null predicate mistake (#14866)

2022-12-07 20:14:13 +08:00

brown_p2/sql

Revert "[test](pipeline) Run nereids cases in p1/p2 (#16130)" (#16792)

2023-02-17 18:48:27 +08:00

compaction

[fix](datatype) fix some bugs about data type array datetimev2 and decimalv3 (#16132)

2023-01-29 14:26:08 +08:00

connector_p0/spark_connector

[regression-test](spark-connector) Add the regression case of the spark doris connector (#14877)

2023-01-18 16:41:41 +08:00

correctness

[fix](mark join) fix bug of mark join with other conjuncts (#16655)

2023-02-14 14:47:15 +08:00

correctness_p0

[fix](planner) the project expr should be calculated in join node in some case (#17035)

2023-02-24 15:20:05 +08:00

csv_header_p0

[Regression](datev2) Add test cases for datev2/datetimev2 (#11831)

2022-08-19 10:57:55 +08:00

data_model_p0

[Enhancement](storage) add a new hidden column __DORIS_VERSION_COL__ for unique key table (#16509)

2023-02-23 15:47:17 +08:00

datatype_p0

[Enhancement](datetimev2-enhance) support 'microseconds_sub' function for datetimev2 (#17130)

2023-02-27 08:47:30 +08:00

datatype_p2/scalar_types/sql

[opt](test) scalar_types_p0 use 100k lines dataset and scalar_types_p2 use 1000k (#16104)

2023-01-19 22:59:29 +08:00

datev2

[fix-core](block) clear block row_same_bit when block reuse (#16172)

2023-02-10 12:21:27 +08:00

ddl_p0

[improvement](meta) make database,table,column names to support unicode (replace PR #13467 with this) (#14531)

2023-02-28 18:50:36 +08:00

decimalv3

[DataType](Deciamlv3) change the avg function scale of decimalv3 (#15445)

2022-12-30 00:27:51 +08:00

delete_p0

[fix](delete) fix delete from bug which can get wrong result (#17146)

2023-02-28 09:20:10 +08:00

demo_p0

[Bug](sort)fix merge sorter might div zero when block bytes less than block rows (#15859)

2023-01-13 18:33:40 +08:00

dynamic_table_p0

[regression-test](dynamic-table) add regression test for dynamic table (#16656)

2023-02-14 00:03:19 +08:00

es_p0

[Fix](DOE)Fix be core dump when parse es epoch_millis date format (#17100)

2023-02-28 20:09:35 +08:00

export

[Feature](map-type) Support stream load and fix some bugs for map type (#16776)

2023-02-19 15:11:54 +08:00

export_p0

[Improvement](outfile) Support output null in parquet writer (#12970)

2022-09-29 13:36:30 +08:00

external_catalog_p0

[Fix](multi catalog)Fix partition case bug (#16763)

2023-02-16 15:47:23 +08:00

external_table_emr_p2

[Fix](multi catalog)Support hive default partition. (#17179)

2023-02-28 00:08:29 +08:00

flink_connector_p0

[regression](flink)add flink doris connector case (#15676)

2023-01-10 17:25:06 +08:00

github_events_p2/sql

Revert "[test](pipeline) Run nereids cases in p1/p2 (#16130)" (#16792)

2023-02-17 18:48:27 +08:00

index_p0

[feature](inverted index)WIP inverted index api: SQL syntax and metadata (#13430)

2022-11-08 23:46:53 +08:00

insert_p0

[fix](struct-type) fix struct subtype support (#17081)

2023-02-28 11:37:07 +08:00

inverted_index_p0

[enhancement](inverted index) Support inverted index without specified parser to use match query (#17110)

2023-02-24 20:34:55 +08:00

inverted_index_p1/tpcds_sf1_index/sql

[Feature-WIP](inverted index) Filter out and remain predicates that do not support apply by inverted index, and add inverted index regression case (#16167)

2023-01-30 22:16:08 +08:00

javaudf_p0

[vectorized](udf) java udf support array type (#16841)

2023-02-20 10:00:25 +08:00

jdbc_catalog_p0

[Improvement](multi catalog)(nereids)Support JDBC external table for new planner. (#17063)

2023-02-28 09:43:04 +08:00

jdbc_p0

[fix](multi-catalog) fix image loading failture when create catalog with resource (#15692)

2023-01-09 09:56:26 +08:00

jsonb_p0

[Bugfix](Jsonb) fix jsonb load in unique key model (#14958)

2022-12-09 21:10:14 +08:00

load/insert

[Feature](map-type) Support stream load and fix some bugs for map type (#16776)

2023-02-19 15:11:54 +08:00

load_p0

[Fix](MySQLLoad) Fix load a big local file bug since bytebuffer from mysql packet using the same byte array (#16901)

2023-02-28 00:06:44 +08:00

map_p0

[Feature](map-type) Support stream load and fix some bugs for map type (#16776)

2023-02-19 15:11:54 +08:00

materialized_view_p0

[Chore](regression-test) add createMV action && add some mv case from fe ut MaterializedViewFunctionTest (#16825)

2023-02-24 16:35:37 +08:00

mtmv_p0

[regression-test](MTMV) Make the case test_create_mtmv more robust (#15866)

2023-01-13 00:13:24 +08:00

mysql_fulltext

[Feature-WIP](inverted index) inverted index reader's implementation, and add mysql_fulltext regression case to test fulltext query (#15823)

2023-01-17 09:13:56 +08:00

nereids_datev2_p1/tpch

[fix](nereids) fix bug in multiply return data type (#15949)

2023-01-20 11:44:24 +08:00

nereids_function_p0

[regression-test](Nereids) add agg function, tvf, generator, window function test cases (#16824)

2023-02-28 17:51:39 +08:00

nereids_p0

[fix](Nereids) json object and json array should always not nullable (#17205)

2023-02-28 20:26:21 +08:00

nereids_syntax_p0

[enhancement](Nereids) datetime v2 type precision derive (#17079)

2023-02-26 22:33:55 +08:00

nereids_tpch_p0/tpch

[refactor](Nereids) let type coercion same with legacy planner (#16844)

2023-02-22 17:29:37 +08:00

opensky_p2/sql

[enhancement](test) add opensky cases to p2 (#12693)

2022-09-19 08:38:17 +08:00

partition_p0

[fix](value-range) fix the value range of non-nullable column contains null causes query short key index error. (#16943)

2023-02-28 11:15:32 +08:00

performance_p0

[enhance](planner)convert 'or' into 'in-predicate' (#15737)

2023-01-18 12:33:20 +08:00

point_query_p0

[Improve](row-store) support row cache (#16263)

2023-02-06 11:16:39 +08:00

primary_index

[fix](merge-on-write) fix that be may coredump when sequence column is null (#16832)

2023-02-20 16:25:52 +08:00

query/regexp

[fix](test) move cases in query to query_p0 (#14452)

2022-11-22 21:35:18 +08:00

query_p0

[vectorized](bug) fix window function can't handle first row of beyond (#17084)

2023-02-28 17:30:23 +08:00

query_p1/lateral_view

[Bug](load) fix core dump on big block load (#13014)

2022-10-10 12:38:32 +08:00

rollup

[Feature](Materialized-View) support advanced Materialized-View (#15212)

2023-01-09 09:53:11 +08:00

rollup_p0

[Bugfix] (ROLLUP) fix the coredump when add rollup by link schema change (#15654)

2023-01-14 10:20:07 +08:00

schema_change

[regression](vectorized) delete vectorized config in regression tests (#15126)

2022-12-16 17:08:29 +08:00

schema_change_p0

[Feature](Materialized-View) forbiden rename column on materialized view (#17030)

2023-02-24 21:28:31 +08:00

segcompaction_p1

segcompaction featuring verticalcompaction (#16731)

2023-03-01 10:55:40 +08:00

sql_block_rule_p0

[fix](test) try to let cases run in parallel (#13114)

2022-10-04 20:56:22 +08:00

ssb_sf0.1_p1/sql

[test](Nereids) add ssb sf0.1 p1 regression case (#17046)

2023-02-23 12:25:10 +08:00

ssb_sf1_p2/sql

[enhancement](regression) split ssb sf1 to sf0.1 to get smaller test data size (#14437)

2022-11-22 10:36:12 +08:00

ssb_sf100_p2/sql

Revert "[test](pipeline) Run nereids cases in p1/p2 (#16130)" (#16792)

2023-02-17 18:48:27 +08:00

ssb_unique_sql_zstd_p0/sql

[improvement](test) group some cases and group a case to p0 if it is not grouped (#11548)

2022-08-06 15:12:08 +08:00

statistics

[feature-wip](statistics) update cache when analysis job finished (#14370)

2022-11-22 21:33:10 +08:00

tpcds_sf1_p1/sql

[fix](planner) cannot process more than one subquery in disjunct (#16506)

2023-02-08 18:46:06 +08:00

tpcds_sf1_unique_p1/sql

Revert "[test](pipeline) Run nereids cases in p1/p2 (#16130)" (#16792)

2023-02-17 18:48:27 +08:00

tpcds_sf100_p2/sql

[Chore](regression) Fix wrong result for decimal (#13644)

2022-10-26 09:24:46 +08:00

tpcds_sf1000_p2/sql

[enhancement](test) add tpcds_sf1000 to p2 (#12695)

2022-09-27 17:12:52 +08:00

tpch_sf0.1_p1/sql

[refactor](regression) make nereids case in a separate dir (#14990)

2022-12-13 16:28:41 +08:00

tpch_sf0.1_unique_p1/sql

Revert "[test](pipeline) Run nereids cases in p1/p2 (#16130)" (#16792)

2023-02-17 18:48:27 +08:00

tpch_sf1_p0/multi_catalog_query

[improvement](test) add tpch1 orc for hive catalog and refactor some test dir (#14669)

2022-11-30 10:03:58 +08:00

tpch_sf1_p1

[fix](test) tpch_sf1_p1 and tpch_sf1_p1/tpch_sf1 are confusing (#14206)

2022-11-28 19:30:32 +08:00

tpch_sf1_p2

rename tpch dir (#14668)

2022-11-30 17:59:13 +08:00

tpch_sf1_unique_p1/sql

[fix](test) tpch_sf1_p1 and tpch_sf1_p1/tpch_sf1 are confusing (#14206)

2022-11-28 19:30:32 +08:00

tpch_sf1_unique_p2/sql

rename tpch dir (#14668)

2022-11-30 17:59:13 +08:00

tpch_sf100_p2/sql

[fix](test) add tpch_sf100 and fix results of tpcds_sf100 (#13098)

2022-10-01 20:53:04 +08:00

tpch_sf100_unique_sql_p2/sql

Revert "[test](pipeline) Run nereids cases in p1/p2 (#16130)" (#16792)

2023-02-17 18:48:27 +08:00

tpch_unique_sql_zstd_p0/sql

[regression-test](unique) Add tpch sf0.1 test under unique table with zstd (#12122)

2022-08-27 21:39:09 +08:00

trino_p0

[fix](planner) fix bug in agg on constant column (#16442)

2023-02-13 11:26:08 +08:00

types/complex_types

[enhancement](test) add more p0 cases (#12285)

2022-09-29 10:45:17 +08:00

update

[fix](update) Fix where clause is not reanalyzed after rewrite (#11723)

2022-08-15 13:24:57 +08:00

with_clause_p0/sql

[enhancement](test) add more p0 cases (#12285)

2022-09-29 10:45:17 +08:00

yandex_metrica_p2

Revert "[test](pipeline) Run nereids cases in p1/p2 (#16130)" (#16792)

2023-02-17 18:48:27 +08:00