66b14f4db1
[fix](mtmv)fix can not create mtmv all use default value ( #28922 )
2023-12-23 21:27:01 +08:00
0a1d9f4cbc
[feature](mtmv)add more test case1 ( #28910 )
2023-12-23 14:39:44 +08:00
623257d02b
[feature](mtmv)MTMV pause and resume ( #28887 )
...
- PAUSE MATERIALIZED VIEW JOB ON mv1
- RESUME MATERIALIZED VIEW JOB ON mv1
- fix when drop db,not drop job
- add lock for one materialized view can only run one task at a time
2023-12-23 14:30:54 +08:00
3d2b4ae244
[fix](mtmv) fix failed to specify the number of buckets when bucket auto ( #28854 )
...
Issue Number: close #xxx
- fix failed to specify the number of buckets when bucket auto
- delete unused SessionVariable
- if mtmv used external table ,check `isMaterializedViewRewriteEnableContainForeignTable`
2023-12-23 09:26:16 +08:00
ae64afd112
[fix](pipelineX) fix cannot runtime obtain profile on pipelineX #28795
2023-12-23 08:24:04 +08:00
e0bb7fa935
[exec](load) change default parallel num from 1 to 8 in no pipeline exec engine ( #28864 )
2023-12-23 00:05:44 +08:00
966766f3b7
[enhancement](broker-load) fix-move-memtable-session-var-for-s3 ( #28894 )
2023-12-22 23:25:06 +08:00
726a9b96c2
[enhancement](udf) add prepare function for java-udf ( #28750 )
2023-12-22 22:15:59 +08:00
3eec62e1d1
[fix](multi-catalog)filter impala generated path ( #28786 )
...
file impala generated dir _imapala_insert_staging
2023-12-22 21:58:50 +08:00
fa0ad56817
[exec](compress) use FragmentTransmissionCompressionCodec control the exchange compress behavior ( #28818 )
2023-12-22 19:50:57 +08:00
b2b209e938
[feature](Nereids) Pull up join from union all ( #28682 )
2023-12-22 19:35:20 +08:00
3ed82bcee2
[Feature](inverted index) add lowercase option for inverted index analyzer ( #28704 )
2023-12-22 18:22:44 +08:00
cb61a0727f
[bug](sharedscan) Fix shared scan bug ( #28841 )
2023-12-22 15:13:14 +08:00
7f310cec9c
[opt](sessionVar)show changed sessoin var first #28840
...
“show variables” command list changed vars before not changed vars,
2023-12-22 14:45:33 +08:00
007f498f3b
(enhance)(InternalQuery) Support to collect profile for intenal query ( #28762 )
2023-12-22 14:03:48 +08:00
f38e11ec4e
[fix](paimon)fix type convert for paimon ( #28774 )
...
fix type convert for paimon
2023-12-22 13:18:25 +08:00
fb52e111c5
[Enhancement](job) No need to query some backends which are not alive. ( #28608 )
...
No need to execute some jobs if backend is not alive
2023-12-22 13:14:30 +08:00
dc786114a2
[Fix](multi-catalog) skip hms events if hms table is not supported. ( #28644 )
...
Co-authored-by: wangxiangyu <wangxiangyu@360shuke.com >
2023-12-22 13:07:58 +08:00
e4c7d4d03a
[chore](error msg) print type info when colocate with ddl failed due to type mismatch ( #28773 )
2023-12-22 12:48:20 +08:00
012e66729a
[improvement](executor) Add tvf and regression test for Workload Scheduler ( #28733 )
...
1 Add select workload schedule policy tvf
2 Add reg test
2023-12-22 12:09:51 +08:00
13ccfa06a7
[Feature](Variant) Implement variant new sub column access method ( #28484 )
...
* [Feature](Variant) Implement variant new sub column access method
The query SELECT v["a"]["b"] from simple_var WHERE cast(v["a"]["b"] as int) = 1 encompasses three primary testing scenarios:
```
1. A basic test involving the variant data type.
2. A scenario dealing with GitHub event data in the context of a variant.
3. A case related to the TPC-H benchmark using a variant.
```
2023-12-22 11:59:37 +08:00
f6b6180462
[Fix](Variant) fix variant predicate rewrite OrToIn with wrong plan ( #28695 )
...
using the name without paths info will lead to wrong In plan, e.g.
```
where cast(v:a as text) = 'hello' or cast(v:b as text) = 'world'
```
will be rewrite to:
```
where cast(v as text) in ('hello', 'world')
``
This is wrong, because they are different slots
2023-12-22 11:51:36 +08:00
0af6bd6390
[fix](group-commit) check if wal need recovery is abnormal ( #28769 )
2023-12-22 11:06:11 +08:00
e2941aa9c8
[improve](config) set mutable and masterOnly in FE config stream_load_default_memtable_on_sink_node ( #28835 )
2023-12-22 10:58:43 +08:00
c1457f97a4
[improvement](external catalog)Change log level to debug for getRowCountFromExternalSource. ( #28801 )
2023-12-21 22:09:50 +08:00
65e3da255f
[enhancement](audit-log) add status code and error msg to audit log for proxy stmt ( #27607 )
2023-12-21 21:16:36 +08:00
e51e94fdec
[fix](Nereids) join order is not right after sql parsing ( #28721 )
...
for sql
```
t1, t2 join t3
```
we should generate plan like:
```
t1 join (t2 join t3)
```
but we generate:
```
(t1 join t2) join t3
```
to follow legancy planner.
2023-12-21 20:31:40 +08:00
0070909d30
[fix](group commit)Fix the issue of duplicate addition of wal path when encouter exception ( #28691 )
2023-12-21 20:27:33 +08:00
ba4033ad2e
[opt](Nereids) improve Shape check cases ( #28124 )
...
* tmplate update
* log tpcds stats when check shape
2023-12-21 20:19:54 +08:00
619d500333
[bug](pipelineX) Fix pipelineX bug on multiple BE ( #28792 )
2023-12-21 19:15:52 +08:00
a16680cf91
[fix](planner)fix bug of bound conjunct to wrong tuple ( #28811 )
...
this fix bug introduced by #28656
2023-12-21 19:15:36 +08:00
6dcf79fc27
[Bug fix][metrics] correct fe collector type for jvm_gc ( #28784 )
...
Co-authored-by: 胥剑旭 <xujianxu@xujianxudeMacBook-Pro.local >
2023-12-21 16:21:13 +08:00
4ddef31c20
(fix)[meta][export] fix replay export NPE issue ( #28752 )
...
The ConnectionContext does not exist in replay thread
2023-12-21 14:39:00 +08:00
4ee661202e
[improve](transaction) extend abort transaction time ( #28662 )
2023-12-21 14:01:05 +08:00
d11bb11592
[enhancement](backup/restore) support alter s3 repo info about ak/sk/token ( #27027 )
...
in some cases:
s3.session_token/AWS_TOKEN will be expired after hours, and may be upload snapshot job will failed if data is big large;
as same reason, repo will be expired too when RepositoryMgr exec repo ping
so it need support alter s3 repo properties about ak/sk/token and update upload snapshot job properties to continue backup.
Signed-off-by: nextdreamblue <zxw520blue1@163.com >
2023-12-21 13:21:55 +08:00
223454d1db
[feature](Nereids) elimite inner join by foreign key ( #28486 )
2023-12-21 12:58:55 +08:00
e9848066c9
[FIX](type) fix matchExactType for complex type ( #28233 )
...
fe matchExactType function should call type.matchTypes for its own logic, do not switch case to do special logic otherwise we may meet core in be like this.
```
F20231208 18:54:39.359673 680131 block.h:535] Check failed: _data_types[i]->is_nullable() target type: Struct(l_info:Nullable(Array(Nullable(String)))) src type: Struct(col:Nullable(Array(Nullable(UInt8))))
*** Check failure stack trace: ***
@ 0x5584e952b926 google::LogMessage::SendToLog()
@ 0x5584e9527ef0 google::LogMessage::Flush()
@ 0x5584e952c169 google::LogMessageFatal::~LogMessageFatal()
@ 0x5584cf17201e doris::vectorized::MutableBlock::merge_impl<>()
@ 0x5584ceac4b1d doris::vectorized::MutableBlock::merge<>()
@ 0x5584d4dd7de3 doris::vectorized::VUnionNode::get_next_const()
@ 0x5584d4dd9a45 doris::vectorized::VUnionNode::get_next()
@ 0x5584bce469bd std::__invoke_impl<>()
@ 0x5584bce466d0 std::__invoke<>()
@ 0x5584bce465c7 _ZNSt5_BindIFMN5doris8ExecNodeEFNS0_6StatusEPNS0_12RuntimeStateEPNS0_10vectorized5BlockEPbEPS1_St12_PlaceholderILi1EESC_ILi2EESC_ILi3EEEE6__callIS2_JOS4_OS7_OS8_EJLm0ELm1ELm2ELm3EEEET_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE
@ 0x5584bce46358 std::_Bind<>::operator()<>()
@ 0x5584bce46208 std::__invoke_impl<>()
@ 0x5584bce46178 _ZSt10__invoke_rIN5doris6StatusERSt5_BindIFMNS0_8ExecNodeEFS1_PNS0_12RuntimeStateEPNS0_10vectorized5BlockEPbEPS3_St12_PlaceholderILi1EESD_ILi2EESD_ILi3EEEEJS5_S8_S9_EENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EESL_E4typeEOSM_DpOSN_
@ 0x5584bce45c18 std::_Function_handler<>::_M_invoke()
@ 0x5584bce6412f std::function<>::operator()()
@ 0x5584bce56382 doris::ExecNode::get_next_after_projects()
@ 0x5584bce26218 doris::PlanFragmentExecutor::get_vectorized_internal()
@ 0x5584bce2431b doris::PlanFragmentExecutor::open_vectorized_internal()
@ 0x5584bce22a96 doris::PlanFragmentExecutor::open()
@ 0x5584bce27c9d doris::PlanFragmentExecutor::execute()
@ 0x5584bcbdb3f8 doris::FragmentMgr::_exec_actual()
@ 0x5584bcbf982f doris::FragmentMgr::exec_plan_fragment()::$_0::operator()()
@ 0x5584bcbf9715 std::__invoke_impl<>()
@ 0x5584bcbf96b5 _ZSt10__invoke_rIvRZN5doris11FragmentMgr18exec_plan_fragmentERKNS0_23TExecPlanFragmentParamsERKSt8functionIFvPNS0_12RuntimeStateEPNS0_6StatusEEEE3$_0JEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EESH_E4typeEOSI_DpOSJ_
@ 0x5584bcbf942d std::_Function_handler<>::_M_invoke()
@ 0x5584b9dfd883 std::function<>::operator()()
@ 0x5584bd6e3929 doris::FunctionRunnable::run()
@ 0x5584bd6cf8ce doris::ThreadPool::dispatch_thread()
```
2023-12-21 11:49:05 +08:00
12089cda08
[refactor](nereids) make NormalizeAggregate rule more clear and readable ( #28607 )
2023-12-21 11:42:22 +08:00
9d1a0b5612
[improve](rpc) Log channel state before shutdown backend service client ( #28667 )
2023-12-21 10:40:56 +08:00
42c5dba9df
[fix](fe ut) fix PropertyConverterTest ( #28722 )
2023-12-21 10:39:54 +08:00
5bf77c8f74
[Fix] (schema change) forbid adding time type column ( #28751 )
2023-12-21 10:27:50 +08:00
e4a58b7407
[Feature](materialized-view) support match logicalAggregate(logicalProject(logicalFilter(logicalOlapScan())) without agg ( #28747 )
...
support match logicalAggregate(logicalProject(logicalFilter(logicalOlapScan())) without agg
2023-12-21 10:27:33 +08:00
8759bce426
[fix](stmt):fix CreateTableStmt toSql placed comment in wrong place ( #27504 )
...
Issue Number: close #27474
Co-authored-by: tongyang.han <tongyang.han@jiduauto.com >
2023-12-21 09:57:20 +08:00
a443a39e2c
[enhance](blacklist) seperate blacklist conf from heartbeat ( #28638 )
...
There is a circuit breaker lasting for 2 minutes in grpc, then if a be is down and up again, send fragments to the be fails lasting for 2 minutes.
2023-12-21 00:17:45 +08:00
a8dcca98ec
[FIX](explode)fix explode array decimal ( #28744 )
...
* fix explode with array<decimal> has specific precision at old planner
2023-12-20 20:19:56 +08:00
280a01b815
[pipelineX](improvement) Support global runtime filter ( #28692 )
2023-12-20 20:06:26 +08:00
ea646c6682
[fix](mtmv)fix thread local reference to checkpoint's Env, causing Env to be unable to be reclaimed, resulting in excessive memory usage by FE ( #28723 )
...
when replay addTaskResult log,will create one ConnectContext,and set Env.getCurrentEnv,then store this ctx in ConnectContext.threadLocalInfo,threadLocalInfo is static,so this ctx can not be recycling，Env of replay thread also can not be recycling
2023-12-20 19:46:58 +08:00
49eed98c1e
[fix](tvf)Fixed the avro-scanner projection pushdown failing to query on multiple BEs ( #28709 )
2023-12-20 19:39:26 +08:00
40afee0097
[fix](Nereids) delete partition failed ( #28717 )
...
1. parser's partitionSpec changed unexpectly by PR #26492
2. delete without using should support un-equals expression
2023-12-20 19:31:53 +08:00
9a5ec43f05
[fix](nereids) Fix data wrong using mv rewrite and ignore case when getting mv related partition table ( #28699 )
...
1. Fix data wrong using mv rewrite
2. Ignore case when getting mv related partition table
3. Enable infer expression column name without alias when create mv
2023-12-20 17:59:46 +08:00
