e51e94fdec
[fix](Nereids) join order is not right after sql parsing ( #28721 )
...
for sql
```
t1, t2 join t3
```
we should generate plan like:
```
t1 join (t2 join t3)
```
but we generate:
```
(t1 join t2) join t3
```
to follow legancy planner.
2023-12-21 20:31:40 +08:00
ba4033ad2e
[opt](Nereids) improve Shape check cases ( #28124 )
...
* tmplate update
* log tpcds stats when check shape
2023-12-21 20:19:54 +08:00
4f1aebb8e8
(topN)runtime_predicate is only triggered when the column name is obtained ( #28419 )
...
Issue Number: close #27485
2023-12-21 18:08:23 +08:00
5d8c465644
[regression](p2) fix test cases result ( #28768 )
...
regression-test/data/external_table_p2/hive/test_hive_hudi.out
regression-test/data/external_table_p2/hive/test_hive_to_array.out
regression-test/suites/external_table_p2/tvf/test_local_tvf_compression.groovy
regression-test/suites/external_table_p2/tvf/test_path_partition_keys.groovy
regression-test/data/external_table_p2/hive/test_hive_text_complex_type.out
2023-12-21 14:38:30 +08:00
e9848066c9
[FIX](type) fix matchExactType for complex type ( #28233 )
...
fe matchExactType function should call type.matchTypes for its own logic, do not switch case to do special logic otherwise we may meet core in be like this.
```
F20231208 18:54:39.359673 680131 block.h:535] Check failed: _data_types[i]->is_nullable() target type: Struct(l_info:Nullable(Array(Nullable(String)))) src type: Struct(col:Nullable(Array(Nullable(UInt8))))
*** Check failure stack trace: ***
@ 0x5584e952b926 google::LogMessage::SendToLog()
@ 0x5584e9527ef0 google::LogMessage::Flush()
@ 0x5584e952c169 google::LogMessageFatal::~LogMessageFatal()
@ 0x5584cf17201e doris::vectorized::MutableBlock::merge_impl<>()
@ 0x5584ceac4b1d doris::vectorized::MutableBlock::merge<>()
@ 0x5584d4dd7de3 doris::vectorized::VUnionNode::get_next_const()
@ 0x5584d4dd9a45 doris::vectorized::VUnionNode::get_next()
@ 0x5584bce469bd std::__invoke_impl<>()
@ 0x5584bce466d0 std::__invoke<>()
@ 0x5584bce465c7 _ZNSt5_BindIFMN5doris8ExecNodeEFNS0_6StatusEPNS0_12RuntimeStateEPNS0_10vectorized5BlockEPbEPS1_St12_PlaceholderILi1EESC_ILi2EESC_ILi3EEEE6__callIS2_JOS4_OS7_OS8_EJLm0ELm1ELm2ELm3EEEET_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE
@ 0x5584bce46358 std::_Bind<>::operator()<>()
@ 0x5584bce46208 std::__invoke_impl<>()
@ 0x5584bce46178 _ZSt10__invoke_rIN5doris6StatusERSt5_BindIFMNS0_8ExecNodeEFS1_PNS0_12RuntimeStateEPNS0_10vectorized5BlockEPbEPS3_St12_PlaceholderILi1EESD_ILi2EESD_ILi3EEEEJS5_S8_S9_EENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EESL_E4typeEOSM_DpOSN_
@ 0x5584bce45c18 std::_Function_handler<>::_M_invoke()
@ 0x5584bce6412f std::function<>::operator()()
@ 0x5584bce56382 doris::ExecNode::get_next_after_projects()
@ 0x5584bce26218 doris::PlanFragmentExecutor::get_vectorized_internal()
@ 0x5584bce2431b doris::PlanFragmentExecutor::open_vectorized_internal()
@ 0x5584bce22a96 doris::PlanFragmentExecutor::open()
@ 0x5584bce27c9d doris::PlanFragmentExecutor::execute()
@ 0x5584bcbdb3f8 doris::FragmentMgr::_exec_actual()
@ 0x5584bcbf982f doris::FragmentMgr::exec_plan_fragment()::$_0::operator()()
@ 0x5584bcbf9715 std::__invoke_impl<>()
@ 0x5584bcbf96b5 _ZSt10__invoke_rIvRZN5doris11FragmentMgr18exec_plan_fragmentERKNS0_23TExecPlanFragmentParamsERKSt8functionIFvPNS0_12RuntimeStateEPNS0_6StatusEEEE3$_0JEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EESH_E4typeEOSI_DpOSJ_
@ 0x5584bcbf942d std::_Function_handler<>::_M_invoke()
@ 0x5584b9dfd883 std::function<>::operator()()
@ 0x5584bd6e3929 doris::FunctionRunnable::run()
@ 0x5584bd6cf8ce doris::ThreadPool::dispatch_thread()
```
2023-12-21 11:49:05 +08:00
12089cda08
[refactor](nereids) make NormalizeAggregate rule more clear and readable ( #28607 )
2023-12-21 11:42:22 +08:00
62833006d6
[test](Nereids): remove shape plan project and distribute in eager test ( #28724 )
2023-12-21 11:14:26 +08:00
34fd376f33
[fix](publish version) fix publish fail but return ok ( #28425 )
2023-12-21 11:10:08 +08:00
e4a58b7407
[Feature](materialized-view) support match logicalAggregate(logicalProject(logicalFilter(logicalOlapScan())) without agg ( #28747 )
...
support match logicalAggregate(logicalProject(logicalFilter(logicalOlapScan())) without agg
2023-12-21 10:27:33 +08:00
a8dcca98ec
[FIX](explode)fix explode array decimal ( #28744 )
...
* fix explode with array<decimal> has specific precision at old planner
2023-12-20 20:19:56 +08:00
d1cc6d9a85
[test](regression-test) order by decs should only make effect on its nearest column #28728
2023-12-20 19:06:41 +08:00
9a5ec43f05
[fix](nereids) Fix data wrong using mv rewrite and ignore case when getting mv related partition table ( #28699 )
...
1. Fix data wrong using mv rewrite
2. Ignore case when getting mv related partition table
3. Enable infer expression column name without alias when create mv
2023-12-20 17:59:46 +08:00
b631c51e6a
[FIX](regresstest) fix case with double value #28668
...
double value in case has 27 figures after the point, which will make not standable output
2023-12-20 17:26:28 +08:00
08066a55f1
[Enhancement](auto-partition) change the behaviour when insert overwrite an auto partition table #28683
...
If we specific target partition(s) when inserting overwrite an auto partition table,
before:
could create new partition
now:
behalf just like non-auto partition table
2023-12-20 17:19:21 +08:00
2a81d238f5
[test](Nereids): remove shape plan project and distribute in eager test ( #28701 )
2023-12-20 14:41:49 +08:00
bcc32b5b26
[feature](invert index) match_regexp feature added ( #28257 )
2023-12-20 14:30:35 +08:00
3e85797443
[fix](Nereids) should only do bind relation in view analyzer ( #28637 )
2023-12-20 10:47:51 +08:00
4c0080e237
[feat](Nereids) support outer join and aggregate bitmap rewrite by mv ( #28596 )
...
- Support left outer join rewrite by materialized view
- Support bitmap_union roll up to imp count(distinct)
- Support partition materialized view rewrite
2023-12-20 10:23:30 +08:00
17268346d3
[Revert](partial update) Revert "Fix missing rowsets during doing alignment when flushing memtable due to compaction ( #28062 )" ( #28674 )
...
This reverts commit 485d7db5161efb011b43a10237df1ca83d5e5c3d.
2023-12-20 10:13:43 +08:00
7da86c37ec
[fix](hive) add support for quoteChar and seperatorChar for hive ( #28613 )
...
add support for quoteChar and seperatorChar .
2023-12-19 19:35:03 +08:00
e362bf674f
[regression-test](memtable) test memtable flush is high priority for vtable writerV1 ( #28502 )
2023-12-19 12:33:12 +08:00
1922638004
[Improvement](regression) change compound predicate regression case name to make it more clear ( #28612 )
2023-12-19 12:17:39 +08:00
ddba98159e
[feature](Nereids) support datev1 and datetimev1 ( #28581 )
2023-12-19 11:31:28 +08:00
88f0bee722
[Performance](point query)Opimize partition prune for point query ( #28150 )
...
* [Performance](point query)Opimize partition prune for point query
2023-12-19 11:00:13 +08:00
6503aaf7db
[feature](planner) allow HLL and QUANTILE_STATE types on duplicate and unique table ( #28546 )
2023-12-19 09:54:24 +08:00
9cf9d568c7
[feature](Nereids): add is null in predicate and put or expansion rule in rewriter ( #28348 )
...
add is null in predicate
put or expansion rule in rewrite job
2023-12-18 20:00:32 +08:00
9ebacb1faa
[fix](expr) fix performance problem caused by too many virtual function call ( #28508 )
2023-12-18 12:01:55 +08:00
fbe5a7c244
[improvement](decimalv2) support check overflow for decimalv2 arithmetics ( #28456 )
2023-12-18 10:54:25 +08:00
1e08845fc5
[regression test](broker load) add case for sequence col ( #27583 )
2023-12-16 22:47:20 +08:00
61de49c727
[case](regression) Test duplicated load id ( #28251 )
...
Co-authored-by: qinhao <qinhao@newland.com.cn >
2023-12-16 22:41:51 +08:00
8ab63a9434
[regression-test][memtable] test memtable flush is high priority for vtable writerV2 ( #28503 )
2023-12-16 22:29:06 +08:00
8c05f7a784
[refactor](cluster)(step-4) remove cluster related to Database ( #27861 )
...
Issue Number: #19897
Remove `default_cluster` prefix related to database.
When upgrading, all prefix will be removed.
2023-12-16 18:28:53 +08:00
a3e2c6affe
[fix](jdbc catalog) fix JdbcScanNode NOT CompoundPredicate filter expr handling errors ( #28497 )
2023-12-16 12:54:55 +08:00
8986bb6bb4
[fix](Planner): parse more Punctuation Date/DateTime ( #28432 )
...
parse more Punctuation as separator, like `2021@01@01 00/00/00`;
2023-12-15 16:17:44 +08:00
0f25a4b3c6
[bug](json)Fix the problem of be down caused by json path ending with \ ( #28180 )
2023-12-15 15:57:08 +08:00
088bb80a9c
[fix](test) fix case of test_unique_table_new_sequence ( #28442 )
...
pr #28105 and #28031 merging conflict caused this case to fail.
2023-12-15 15:10:39 +08:00
97b033813a
[perf](Nereids) add back canEliminate temporary ( #28017 )
2023-12-15 14:26:29 +08:00
4c51558f6b
[feature](nereids) Support basic aggregate rewrite and function rollup using materialized view ( #28269 )
...
Add aggregate materializedviewRules for query rewrite.
it support the query rewrite as following:
def mv = "select lineitem.L_LINENUMBER, orders.O_CUSTKEY, sum(O_TOTALPRICE) as sum_alias " +
"from lineitem " +
"inner join orders on lineitem.L_ORDERKEY = orders.O_ORDERKEY " +
"group by lineitem.L_LINENUMBER, orders.O_CUSTKEY "
def query = "select lineitem.L_LINENUMBER, sum(O_TOTALPRICE) as sum_alias " +
"from lineitem " +
"inner join orders on lineitem.L_ORDERKEY = orders.O_ORDERKEY " +
"group by lineitem.L_LINENUMBER"
2023-12-15 11:30:02 +08:00
ce60064573
[regression-test](variant) fix unstable query result m… ( #28390 )
...
* [regression-test](variant) fix unstable query result for unique key model
* fix p2 case
* add msg
2023-12-15 10:52:50 +08:00
4d9b6c272d
[Fix](vcompound pred) Corrected evaluation for compound predicates with constant columns ( #28421 )
2023-12-15 10:10:48 +08:00
eb99e4270d
[Fix](parquet_reader) Fix dict filtering doesn't work with plain dict encoding in parquet reader. ( #28290 )
2023-12-15 09:27:02 +08:00
8ca7bd8f98
[enhancement](bitmap)support bitmap type for non-key column in duplicate table ( #28392 )
2023-12-14 23:59:12 +08:00
4300fdce43
[feature](merge-on-write) enable merge-on-write by default again ( #28105 )
...
fix #27188 , #28096
2023-12-14 19:35:52 +08:00
a540b7eb43
[fix](Nereids) multi_distinct_sum should inherit NullableAggregateFunction ( #28349 )
2023-12-14 19:27:06 +08:00
01c94a554d
[fix](autoinc) Fix broker load when target table has autoinc column ( #28402 )
2023-12-14 18:02:54 +08:00
c08ab9edc7
[feature](HiveCatalog) Support for getting hive meta data from relational databases under HMS ( #28188 )
2023-12-14 17:50:17 +08:00
6c9c1ef2f2
[test](regression) add flink sync db case ( #19328 )
...
add flink sync db case
Mock cdc sync data write
Contains multi-table import, add and drop columns
2023-12-14 17:06:17 +08:00
ee24667b9f
[fix](group commit) Fix some group commit problems ( #28319 )
2023-12-14 14:38:56 +08:00
e53cfa09da
[fix](join) incorrect result of right anti join with nullable ( #28301 )
2023-12-14 14:07:12 +08:00
bb23078d43
[feture](Nereids) support delete from without using ( #28083 )
...
support sql: DELETE FROM <table_name> WHERE <predicates>
2023-12-14 13:54:33 +08:00
