Commit Graph

3430 Commits

Author SHA1 Message Date
ddba98159e [feature](Nereids) support datev1 and datetimev1 (#28581) 2023-12-19 11:31:28 +08:00
88f0bee722 [Performance](point query)Opimize partition prune for point query (#28150)
* [Performance](point query)Opimize partition prune for point query
2023-12-19 11:00:13 +08:00
7b7845e695 [fix](planner) choice wrong length of string type output of union (#28514) 2023-12-19 10:43:39 +08:00
f9ddf8c7ef [improvement](be report) add be report http (#28424) 2023-12-19 10:39:19 +08:00
Pxl
d6514618b2 [Improvement](decimal) reduce overhead on disable check decimal overflow (#28249)
reduce overhead on disable check decimal overflow
2023-12-19 10:12:30 +08:00
6503aaf7db [feature](planner) allow HLL and QUANTILE_STATE types on duplicate and unique table (#28546) 2023-12-19 09:54:24 +08:00
66fbb22ad7 [fix](group commit) Fix some wal problems on group commit (#28554) 2023-12-19 09:51:03 +08:00
9cf9d568c7 [feature](Nereids): add is null in predicate and put or expansion rule in rewriter (#28348)
add is null in predicate
put or expansion rule in rewrite job
2023-12-18 20:00:32 +08:00
b38286c398 [Fix](regresson) fix vcompound constant column regression case (#28560) 2023-12-18 18:18:58 +08:00
a267a7fbe3 [fix](arrow-flight) Modify regression test Arrow version to 14.0.1 (#28472)
Same as #28093
2023-12-18 14:36:59 +08:00
9ebacb1faa [fix](expr) fix performance problem caused by too many virtual function call (#28508) 2023-12-18 12:01:55 +08:00
fbe5a7c244 [improvement](decimalv2) support check overflow for decimalv2 arithmetics (#28456) 2023-12-18 10:54:25 +08:00
d11365da9c [Fix](memtable) fix shrink_memtable_by_agg should also update _row_in_blocks (#28536)
Otherwise using the stale `_row_in_blocks` will result in heap-buffer-overflow

```
==2695213==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x62900122e210 at pc 0x56524744aecf bp 0x7f62c595ef7
0 sp 0x7f62c595ef68
READ of size 8 at 0x62900122e210 thread T1627 (MemTableFlushTh)
    #0 0x56524744aece in doris::vectorized::ColumnVector<long>::insert_indices_from(doris::vectorized::IColumn const&, unsigned int const*, unsigned int const*) /mnt/disk2/lihangyu/doris/be/src/vec/columns/column_vector.cpp:378:33
    #1 0x5652472a7538 in doris::vectorized::ColumnNullable::insert_indices_from(doris::vectorized::IColumn const&, unsigned int const*, unsigned int const*) /mnt/disk2/lihangyu/doris/be/src/vec/columns/column_nullable.cpp:310:25
    #2 0x56524782a62a in doris::vectorized::MutableBlock::add_rows(doris::vectorized::Block const*, unsigned int const*, unsigned int const*) /mnt/disk2/lihangyu/doris/be/src/vec/core/block.cpp:961:14
    #3 0x565233f187ae in doris::MemTable::_put_into_output(doris::vectorized::Block&) /mnt/disk2/lihangyu/doris/be/src/olap/memtable.cpp:248:27
    #4 0x565233f1db66 in doris::MemTable::to_block() /mnt/disk2/lihangyu/doris/be/src/olap/memtable.cpp:496:13
    #5 0x565233efae60 in doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*) /mnt/disk2/lihangyu/doris/be/src/olap/memtable_flush_executor.cpp:121:62
    #6 0x565233efc8d6 in doris::FlushToken::_flush_memtable(doris::MemTable*, int, long) /mnt/disk2/lihangyu/doris/be/src/olap/memtable_flush_executor.cpp:150:16
    #7 0x565233f0c5eb in doris::MemtableFlushTask::run() /mnt/disk2/lihangyu/doris/be/src/olap/memtable_flush_executor.cpp:58:23
```
2023-12-18 10:31:16 +08:00
683c173480 [fix](test) fix index change testcases #28298 2023-12-16 23:42:02 +08:00
1e08845fc5 [regression test](broker load) add case for sequence col (#27583) 2023-12-16 22:47:20 +08:00
61de49c727 [case](regression) Test duplicated load id (#28251)
Co-authored-by: qinhao <qinhao@newland.com.cn>
2023-12-16 22:41:51 +08:00
74c0a3060f [feature](jdk) Using G1 as defaut garbage colletor in FE (#28263) 2023-12-16 22:40:11 +08:00
8ab63a9434 [regression-test][memtable] test memtable flush is high priority for vtable writerV2 (#28503) 2023-12-16 22:29:06 +08:00
8c05f7a784 [refactor](cluster)(step-4) remove cluster related to Database (#27861)
Issue Number: #19897

Remove `default_cluster` prefix related to database.
When upgrading, all prefix will be removed.
2023-12-16 18:28:53 +08:00
608baae001 [docker][regregression]update routine load cases #28450
Co-authored-by: 胥剑旭 <xujianxu@xujianxudeMacBook-Pro.local>
2023-12-16 17:57:39 +08:00
a3e2c6affe [fix](jdbc catalog) fix JdbcScanNode NOT CompoundPredicate filter expr handling errors (#28497) 2023-12-16 12:54:55 +08:00
920c75c870 [fix](ci)tpch pipeline add check (#28370)
Co-authored-by: stephen <hello-stephen@qq.com>
2023-12-16 11:11:28 +08:00
f12a225844 [fix](session variables) Make default value of max_execution_time same to query_timeout #28474
Current problem, UNSET global VARIABLE ALL will write an oplog, which makes query_timeout = 0 when we replay it in a future time-stamp. So we change default value of max_execution_time to 90000 which is consistent to query_timeout default value.
2023-12-16 10:59:05 +08:00
8986bb6bb4 [fix](Planner): parse more Punctuation Date/DateTime (#28432)
parse more Punctuation as separator, like `2021@01@01 00/00/00`;
2023-12-15 16:17:44 +08:00
0f25a4b3c6 [bug](json)Fix the problem of be down caused by json path ending with \ (#28180) 2023-12-15 15:57:08 +08:00
088bb80a9c [fix](test) fix case of test_unique_table_new_sequence (#28442)
pr #28105 and #28031 merging conflict caused this case to fail.
2023-12-15 15:10:39 +08:00
97b033813a [perf](Nereids) add back canEliminate temporary (#28017) 2023-12-15 14:26:29 +08:00
4c51558f6b [feature](nereids) Support basic aggregate rewrite and function rollup using materialized view (#28269)
Add aggregate materializedviewRules for query rewrite.
it support the query rewrite as following:

    def mv = "select lineitem.L_LINENUMBER, orders.O_CUSTKEY, sum(O_TOTALPRICE) as sum_alias " +
            "from lineitem " +
            "inner join orders on lineitem.L_ORDERKEY = orders.O_ORDERKEY " +
            "group by lineitem.L_LINENUMBER, orders.O_CUSTKEY "
    def query = "select lineitem.L_LINENUMBER, sum(O_TOTALPRICE) as sum_alias " +
            "from lineitem " +
            "inner join orders on lineitem.L_ORDERKEY = orders.O_ORDERKEY " +
            "group by lineitem.L_LINENUMBER"
2023-12-15 11:30:02 +08:00
Pxl
8661b5ec21 [Bug](function) fix npe on select http_stream directly (#28423)
fix npe on select http_stream directly
2023-12-15 11:09:45 +08:00
ce60064573 [regression-test](variant) fix unstable query result m… (#28390)
* [regression-test](variant) fix unstable query result for unique key model

* fix p2 case

* add msg
2023-12-15 10:52:50 +08:00
4d9b6c272d [Fix](vcompound pred) Corrected evaluation for compound predicates with constant columns (#28421) 2023-12-15 10:10:48 +08:00
eb99e4270d [Fix](parquet_reader) Fix dict filtering doesn't work with plain dict encoding in parquet reader. (#28290) 2023-12-15 09:27:02 +08:00
415c6d854d [fix](partial update) Fix some bugs about partial update (#28358) 2023-12-15 00:04:29 +08:00
8ca7bd8f98 [enhancement](bitmap)support bitmap type for non-key column in duplicate table (#28392) 2023-12-14 23:59:12 +08:00
4300fdce43 [feature](merge-on-write) enable merge-on-write by default again (#28105)
fix #27188, #28096
2023-12-14 19:35:52 +08:00
a540b7eb43 [fix](Nereids) multi_distinct_sum should inherit NullableAggregateFunction (#28349) 2023-12-14 19:27:06 +08:00
01c94a554d [fix](autoinc) Fix broker load when target table has autoinc column (#28402) 2023-12-14 18:02:54 +08:00
c08ab9edc7 [feature](HiveCatalog) Support for getting hive meta data from relational databases under HMS (#28188) 2023-12-14 17:50:17 +08:00
38e79e32fa [fix](mtmv)fix start time can not be earlier than the current time (#28379) 2023-12-14 17:28:04 +08:00
6c9c1ef2f2 [test](regression) add flink sync db case (#19328)
add flink sync db case

Mock cdc sync data write
Contains multi-table import, add and drop columns
2023-12-14 17:06:17 +08:00
ee24667b9f [fix](group commit) Fix some group commit problems (#28319) 2023-12-14 14:38:56 +08:00
e53cfa09da [fix](join) incorrect result of right anti join with nullable (#28301) 2023-12-14 14:07:12 +08:00
bb23078d43 [feture](Nereids) support delete from without using (#28083)
support sql: DELETE FROM <table_name> WHERE <predicates>
2023-12-14 13:54:33 +08:00
e5a57f82ec [fix](Variant Type) Fixes the desc failure (#28343)
fix the desc failure when there is no decomposition of columns in the variant column.
2023-12-14 13:20:43 +08:00
33e24436d6 [fix](regression) mow cases should use assertEquals not assertTrue (#28338) 2023-12-13 22:14:32 +08:00
a271fee3c5 [test](statistics)Add external empty table test case. (#28267) 2023-12-13 21:48:01 +08:00
099b1b7106 [fix](statistics)Fix column stats trigger info bug (#28303)
Before, we didn't update the jobType info in ColStatsMeta. This will case the jobType always be the type
when it first be set. For example, if we manually analyzed table, the jobType will always be MANUAL,
even if this table is auto analyzed again later.
2023-12-13 20:31:03 +08:00
0868526f6a [fix](sequence column) fix update fail on nereids planner (#28031)
1. if we set enable_fallback_to_original_planner to false, the UPDATE sql in regression case test_unique_table_sequence will fail due to:“Table test_uniq_sequence has sequence column, need to specify the sequence column”,The bug is introduced by [fix](sequence column) insert into should require sequence column in all scenario #27780
2. fix insert fail in transaction mode, which is introduced by [refactor](Nereids) let insert into compatible with legacy planner #27947
3. add cases with session variable enable_fallback_to_original_planner false
2023-12-13 19:41:41 +08:00
dbc2b081d7 [fix](Nereids) could not do partition prunning when predicates is NULL (#28294) 2023-12-13 19:36:42 +08:00
4f5821407f [case]Load data with load_parallelism=any > 1 and stream load with compress type (#27306) 2023-12-13 18:41:14 +08:00