database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
14,559 Commits 1 Branch 0 Tags
1e2a614a46e6bee103ffc35c7becf2bd9d930fe2
Commit Graph

14559 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Adonis Ling
1e2a614a46 [fix](workflow) Fix failure test cases in BE UT (macOS) (#26425) 
1. Fix memory issues in LoadStreamMgrTest.
2. Skip S3FileWriterTest by default because it depends on the environment in teamcity.
3. Fix VTimestampFunctionsTest.convert_tz_test.
 2023-11-06 10:44:44 +08:00
zclllyybb
f226117699 [fix](auto-partition) Fix auto partition concurrent conflict (#26166) 2023-11-06 10:34:26 +08:00
zhangstar333
a5b9d61f43 [bug](udf) fix java-udf function const column without arguments (#26409) 2023-11-06 10:34:09 +08:00
wuwenchi
d9bd1fad67 [fix](spark-load)fix-Unique-key-with-MOR-by-sparkload (#26383) 
When a Unique key table carries the `enable_unique_key_merge_on_write` attribute, the value of the agg type is none. Therefore, when doing sparkload, we need to specify the agg type as `REPLACE`.
 2023-11-06 09:56:46 +08:00
zzzxl
99de6c7afe [fix](invert index) Fix the timing error when opening the searcher (#26401) 2023-11-04 01:52:35 -05:00
amory
6ef2e62600 [FIX](struct) fix struct be nested when read will make core (#26270) 2023-11-03 09:55:39 -05:00
starocean999
11059ca4ed [fix](nereids)SimplifyRange rule may mess up and/or predicate (#26304) 2023-11-03 20:38:07 +08:00
starocean999
646348ccc4 [fix](nereids)add visitMarkJoinReference method in ExpressionDeepCopier (#25874) 2023-11-03 20:34:29 +08:00
starocean999
8c3e173553 [fix](nereids)fix bug of select mv in nereids (#26235) 
* revert some change from pr26192
* disable some case for nereids
 2023-11-03 20:28:55 +08:00
caiconghui
d367d9b585 [fix](show_proc) fix show statistic proc dir to ensure that result only contains dbs in internal catalog (#26254) 2023-11-03 20:12:06 +08:00
Tiewei Fang
7730a9025e [Fix](Regression-test) add test for tvf (#26322) 2023-11-03 19:07:07 +08:00
amory
fd92810df8 [FIX](regresstest)fixed out file for test-cast-map-function test (#26389) 2023-11-03 05:47:10 -05:00
seawinde
b19198b305 [opt](Nereids) Support to get notdeterministic functions and base tables from plan (#26120) 
Support to get notdeterministic functions and  base tables from plan.
These are implemented by Plan Visitor

**table collect useage example as following:**
```
TableCollectorContext collectorContext =
    new PlanVisitors.TableCollectorContext(Sets.newHashSet(TableType.OLAP));
physicalPlan.accept(PlanVisitors.TABLE_COLLECTOR, collectorContext);
```
`collectorContext.getCollectedTables()` has the tables

**nondeterministic functions collect useage example as following:**
```
List<Expression> functionCollectResult = new ArrayList<>();
plan.accept(PlanVisitors.NondeterministicCollector.INSTANCE, collectResult);
```
`functionCollectResult` has the nondeterministic functions
 2023-11-03 04:37:46 -05:00
wangbo
7a0ee04deb [feature](executor)refactor show workload group schema #26100 
2 add query queue info
 2023-11-03 17:34:48 +08:00
zhiqiang
f16256362c [opt](regression test) Add string-like column order by test #26379 2023-11-03 17:31:20 +08:00
jakevin
074323f112 [enhancement](Nereids): rearrange Limit optimization rules (#26378) 2023-11-03 17:13:44 +08:00
谢健
265c83d912 [fix](Planner): don't push down isNull predicate into view (#26288) 
We can't push down `isNull` predicate into view
 2023-11-03 16:30:12 +08:00
zzzxl
f92b572a97 [fix](invert index) optimize SeekTime profile (#26353) 2023-11-03 03:03:29 -05:00
Xinyi Zou
54548728ec 1 (#26363) 2023-11-03 15:41:31 +08:00
Xinyi Zou
867f44d606 [opt](memory) jemalloc conf lg_tcache_max restore default #26362 
tc/jemalloc_free_memory in web beip:8040/mem_tracker is the cache size of Jemalloc.

Previously lg_tcache_max:20, this will cache up to 1M Bin in the thread cache, which will cause the Jemalloc cache to be too large in some scenarios.

Restore the default lg_tcache_max:16, which can cache a maximum of 64K Bins.

If you are doing a performance POC, you can consider increasing it.
 2023-11-03 15:35:38 +08:00
walter
a4637e4462 [fix](fe) Add/remove frontends after updating BEBJE (#26313) 
To ensure exception safety, adding or removing frontends in memory
should execute after removing the electable nodes (might throw
exceptions)
 2023-11-03 14:33:06 +08:00
morrySnow
74fd5da420 [fix](Nereids) time extract function constant folding core (#26292) 
some time extract function changed return type in the previous PR #18369
but it is not change FE constant folding function signature.
This is let them have same signature to avoid BE core.
 2023-11-03 01:24:12 -05:00
morrySnow
a89477e8b5 [fix](Nereids) could not run query with repeat node in cte (#26330) 
ExpressionDeepCopier not process VirtualReference, so we generate inline
plan with mistake.
 2023-11-03 01:24:01 -05:00
starocean999
9243de1898 [fix](planner) add IP type to compatibilityMatrix (#26095) 2023-11-03 01:19:22 -05:00
morrySnow
0f24500ff8 [fix](Nereids) RewriteCteChildren not work with cost based rewritter (#26326) 
we use a map to record rewrite cte children result to avoid rewrite
twice in cost based rewritter. However, we record cte outer and
inner in one map, and use null as outer result's key, use cte id as
inner result's key. This is wrong, because every anchor has an outer,
and we could only record one outer. So when we use the cache in cost
based rewritter, we get wrong outer plan from the cache. Then the error
will be thrown as below:

```
Caused by: java.lang.IllegalArgumentException: Stats for CTE: CTEId#1 not found
    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:143) ~[guava-32.1.2-jre.jar:?]
    at org.apache.doris.nereids.stats.StatsCalculator.visitLogicalCTEConsumer(StatsCalculator.java:1049) ~[classes/:?]
    at org.apache.doris.nereids.stats.StatsCalculator.visitLogicalCTEConsumer(StatsCalculator.java:147) ~[classes/:?]
    at org.apache.doris.nereids.trees.plans.logical.LogicalCTEConsumer.accept(LogicalCTEConsumer.java:111) ~[classes/:?]
    at org.apache.doris.nereids.stats.StatsCalculator.estimate(StatsCalculator.java:222) ~[classes/:?]
    at org.apache.doris.nereids.stats.StatsCalculator.estimate(StatsCalculator.java:200) ~[classes/:?]
    at org.apache.doris.nereids.jobs.cascades.DeriveStatsJob.execute(DeriveStatsJob.java:108) ~[classes/:?]
    at org.apache.doris.nereids.jobs.scheduler.SimpleJobScheduler.executeJobPool(SimpleJobScheduler.java:39) ~[classes/:?]
    at org.apache.doris.nereids.jobs.executor.Optimizer.execute(Optimizer.java:51) ~[classes/:?]
    at org.apache.doris.nereids.jobs.rewrite.CostBasedRewriteJob.getCost(CostBasedRewriteJob.java:98) ~[classes/:?]
    at org.apache.doris.nereids.jobs.rewrite.CostBasedRewriteJob.execute(CostBasedRewriteJob.java:64) ~[classes/:?]
    at org.apache.doris.nereids.jobs.executor.AbstractBatchJobExecutor.execute(AbstractBatchJobExecutor.java:119) ~[classes/:?]
    at org.apache.doris.nereids.rules.rewrite.RewriteCteChildren.visit(RewriteCteChildren.java:72) ~[classes/:?]
    at org.apache.doris.nereids.rules.rewrite.RewriteCteChildren.visit(RewriteCteChildren.java:56) ~[classes/:?]
    at org.apache.doris.nereids.trees.plans.visitor.PlanVisitor.visitLogicalSink(PlanVisitor.java:118) ~[classes/:?]
    at org.apache.doris.nereids.trees.plans.visitor.SinkVisitor.visitLogicalResultSink(SinkVisitor.java:72) ~[classes/:?]
    at org.apache.doris.nereids.trees.plans.logical.LogicalResultSink.accept(LogicalResultSink.java:58) ~[classes/:?]
    at org.apache.doris.nereids.rules.rewrite.RewriteCteChildren.visitLogicalCTEAnchor(RewriteCteChildren.java:86) ~[classes/:?]
    at org.apache.doris.nereids.rules.rewrite.RewriteCteChildren.visitLogicalCTEAnchor(RewriteCteChildren.java:56) ~[classes/:?]
    at org.apache.doris.nereids.trees.plans.logical.LogicalCTEAnchor.accept(LogicalCTEAnchor.java:60) ~[classes/:?]
    at org.apache.doris.nereids.rules.rewrite.RewriteCteChildren.visitLogicalCTEAnchor(RewriteCteChildren.java:86) ~[classes/:?]
    at org.apache.doris.nereids.rules.rewrite.RewriteCteChildren.visitLogicalCTEAnchor(RewriteCteChildren.java:56) ~[classes/:?]
    at org.apache.doris.nereids.trees.plans.logical.LogicalCTEAnchor.accept(LogicalCTEAnchor.java:60) ~[classes/:?]
    at org.apache.doris.nereids.rules.rewrite.RewriteCteChildren.rewriteRoot(RewriteCteChildren.java:67) ~[classes/:?]
    at org.apache.doris.nereids.jobs.rewrite.CustomRewriteJob.execute(CustomRewriteJob.java:58) ~[classes/:?]
    at org.apache.doris.nereids.jobs.executor.AbstractBatchJobExecutor.execute(AbstractBatchJobExecutor.java:119) ~[classes/:?]
    at org.apache.doris.nereids.NereidsPlanner.rewrite(NereidsPlanner.java:275) ~[classes/:?]
    at org.apache.doris.nereids.NereidsPlanner.plan(NereidsPlanner.java:218) ~[classes/:?]
    at org.apache.doris.nereids.NereidsPlanner.plan(NereidsPlanner.java:118) ~[classes/:?]
    at org.apache.doris.nereids.trees.plans.commands.ExplainCommand.run(ExplainCommand.java:81) ~[classes/:?]
    at org.apache.doris.qe.StmtExecutor.executeByNereids(StmtExecutor.java:550) ~[classes/:?]
```
 2023-11-03 01:12:31 -05:00
morrySnow
6892fc99f8 [fix](Nereids) only search internal funcftion when dbName is empty (#26296) 
if call function with database name. we should only search UDF
 2023-11-03 01:01:25 -05:00
Xiangyu Wang
9019a71770 [Fix](multi-catalog) Filter _temporary dir which is generated by spark. (#26194) 
Sometimes there are some temporary files which are generated by some spark jobs, we need filter these files.

```
errCode = 2, detailMessage = (xxx.xxx.xxx.xxx.xxx)
[CANCELLED][INTERNAL_ERROR]failed to init reader for file hdfs://xxx/user/hive/warehouse/xxx.db/xxx/_temporary/0/_temporary/attempt_202311010952379045223848154629697_0013_m_000442_3355/part-00442-3f97ac18-0583-4d2c-8995-8d21ceda56ec-c000.snappy.orc, 
err: [INTERNAL_ERROR]Init OrcReader failed. reason = Failed to parse the postscript from hdfs://xxx/user/hive/warehouse/xxx.db/xxx/_temporary/0/_temporary/attempt_202311010952379045223848154629697_0013_m_000442_3355/part-00442-3f97ac18-0583-4d2c-8995-8d21ceda56ec-c000.snappy.orc   
0.  @ 0x000000000bdbd89f
   doris::Status doris::Status::Error<6, true, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>(std::basic_string_view<char, std::char_traits<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)  /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:173  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  1.  @ 0x000000000eccecfe
   doris::vectorized::OrcReader::_create_file_reader()  /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  2.  @ 0x000000000ecceff9
   doris::vectorized::OrcReader::init_reader(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const*, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::variant<doris::ColumnValueRange<(doris::PrimitiveType)3>, doris::ColumnValueRange<(doris::PrimitiveType)4>, doris::ColumnValueRange<(doris::PrimitiveType)5>, doris::ColumnValueRange<(doris::PrimitiveType)6>, doris::ColumnValueRange<(doris::PrimitiveType)7>, doris::ColumnValueRange<(doris::PrimitiveType)15>, doris::ColumnValueRange<(doris::PrimitiveType)10>, doris::ColumnValueRange<(doris::PrimitiveType)23>, doris::ColumnValueRange<(doris::PrimitiveType)11>, doris::ColumnValueRange<(doris::PrimitiveType)25>, doris::ColumnValueRange<(doris::PrimitiveType)12>, doris::ColumnValueRange<(doris::PrimitiveType)26>, doris::ColumnValueRange<(doris::PrimitiveType)20>, doris::ColumnValueRange<(doris::PrimitiveType)2>, doris::ColumnValueRange<(doris::PrimitiveType)19>, doris::ColumnValueRange<(doris::PrimitiveType)28>, doris::ColumnValueRange<(doris::PrimitiveType)29>, doris::ColumnValueRange<(doris::PrimitiveType)30> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::variant<doris::ColumnValueRange<(doris::PrimitiveType)3>, doris::ColumnValueRange<(doris::PrimitiveType)4>, doris::ColumnValueRange<(doris::PrimitiveType)5>, doris::ColumnValueRange<(doris::PrimitiveType)6>, doris::ColumnValueRange<(doris::PrimitiveType)7>, doris::ColumnValueRange<(doris::PrimitiveType)15>, doris::ColumnValueRange<(doris::PrimitiveType)10>, doris::ColumnValueRange<(doris::PrimitiveType)23>, doris::ColumnValueRange<(doris::PrimitiveType)11>, doris::ColumnValueRange<(doris::PrimitiveType)25>, doris::ColumnValueRange<(doris::PrimitiveType)12>, doris::ColumnValueRange<(doris::PrimitiveType)26>, doris::ColumnValueRange<(doris::PrimitiveType)20>, doris::ColumnValueRange<(doris::PrimitiveType)2>, doris::ColumnValueRange<(doris::PrimitiveType)19>, doris::ColumnValueRange<(doris::PrimitiveType)28>, doris::ColumnValueRange<(doris::PrimitiveType)29>, doris::ColumnValueRange<(doris::PrimitiveType)30> > > > >*, std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std::shared_ptr<doris::vectorized::VExprContext> > > const&, bool, doris::TupleDescriptor const*, doris::RowDescriptor const*, std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std::shared_ptr<doris::vectorized::VExprContext> > > const*, std::unordered_map<int, std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std::shared_ptr<doris::vectorized::VExprContext> > >, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std::shared_ptr<doris::vectorized::VExprContext> > > > > > const*)  /root/doris-master/be/src/common/status.h:448  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  3.  @ 0x000000000ff43da0
   doris::vectorized::VFileScanner::_get_next_reader()  /root/doris-master/be/src/vec/exec/scan/vfile_scanner.cpp:800  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  4.  @ 0x000000000ff4256f
   doris::vectorized::VFileScanner::_get_block_impl(doris::RuntimeState*, doris::vectorized::Block*, bool*)  /root/doris-master/be/src/common/status.h:448  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  5.  @ 0x000000000ffdb1aa
   doris::vectorized::VScanner::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*)  /root/doris-master/be/src/vec/exec/scan/vscanner.cpp:0  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  6.  @ 0x000000000ff3b379
   doris::vectorized::ScannerScheduler::_scanner_scan(doris::vectorized::ScannerScheduler*, doris::vectorized::ScannerContext*, std::shared_ptr<doris::vectorized::VScanner>)  /root/doris-master/be/src/common/status.h:346  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  7.  @ 0x000000000ff3ca11  std::_Function_handler<void (), doris::vectorized::ScannerScheduler::_schedule_scanners(doris::vectorized::ScannerContext*)::$_1::operator()() const::{lambda()#4}>::_M_invoke(std::_Any_data const&)  /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  8.  @ 0x000000000c986608
   doris::ThreadPool::dispatch_thread()  /root/doris-master/be/src/util/threadpool.cpp:0  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  9.  @ 0x000000000c97c2ac
   doris::Thread::supervise_thread(void*)  /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  10. @ 0x0000000000007ea5  start_thread  in /usr/lib64/libpthread-2.17.so  11. @ 0x00000000000feb0d  __clone  in /usr/lib64/libc-2.17.so    0.  @ 0x000000000c647366
   doris::Status doris::Status::Error<6, true, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(std::basic_string_view<char, std::char_traits<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&)  /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:173  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  1.  @ 0x000000000ff42dc3
   doris::vectorized::VFileScanner::_get_next_reader()  /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  2.  @ 0x000000000ff4256f
   doris::vectorized::VFileScanner::_get_block_impl(doris::RuntimeState*, doris::vectorized::Block*, bool*)  /root/doris-master/be/src/common/status.h:448  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  3.  @ 0x000000000ffdb1aa
   doris::vectorized::VScanner::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*)  /root/doris-master/be/src/vec/exec/scan/vscanner.cpp:0  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  4.  @ 0x000000000ff3b379
   doris::vectorized::ScannerScheduler::_scanner_scan(doris::vectorized::ScannerScheduler*, doris::vectorized::ScannerContext*, std::shared_ptr<doris::vectorized::VScanner>)  /root/doris-master/be/src/common/status.h:346  in /hdpData10/yarn/localcache/usercache/hive/appcache/a…
```
Co-authored-by: wangxiangyu <wangxiangyu@360shuke.com>
 2023-11-03 13:42:29 +08:00
Gabriel
3d403b061c [test](coverage) Improve test coverage for runtime filter (#26314) 2023-11-03 12:17:03 +08:00
meiyi
b19f275714 [improvement](insert) refactor group commit insert into (#25795) 2023-11-03 12:02:40 +08:00
zhangdong
61edb9e4d8 [feature](mtmv)(2)Implementing mtmv using antlr (#26102) 
Implementing mtmv using antlr，

No specific business logic was implemented, therefore an exception was thrown during execution
```
throw new AnalysisException("current not support.");
```
 2023-11-03 11:57:00 +08:00
zhangstar333
509723c1d6 [bug](bitmap) fix bitmap value not clear get wrong data (#26321) 2023-11-03 10:25:13 +08:00
xueweizhang
03ab822ba6 [fix](backup) support aws_token for backup/restore (#26242) 2023-11-03 10:18:37 +08:00
Tiewei Fang
91946dc8b0 [minor](serde) 'null' in the serde should be represented by constants, instead of a string variable (#26301) 2023-11-02 23:13:10 +08:00
amory
93a934e775 [Improve](map) support map cast with map literal and implicate nested scala cast (#26126) 2023-11-02 09:56:42 -05:00
airborne12
3f47eb5e89 [Fix](inverted index) fix compound query result error when disable inverted_index_query session variable (#26327) 2023-11-02 08:36:08 -05:00
jakevin
e307884e57 [enhancement](Nereids): GroupPlan don't generate ObjectId (#26315) 2023-11-02 21:35:48 +08:00
Xin Liao
8bd06aff7e [Chore](MoW) remove unused code about rowset tree (#26282) 2023-11-02 20:25:27 +08:00
walter
d9621404b3 [improvement](fe) Add more log about add/remove frontends (#26306) 2023-11-02 20:19:51 +08:00
walter
c098b0393a [test](regression) Add backup/restore DB case (#26312) 2023-11-02 20:18:45 +08:00
xy720
a5ef90dacc [enhancement](recover) support skipping missing version in select by session variable (#25654) 2023-11-02 20:01:51 +08:00
luozenglin
89cf828f03 [enhancement](udf) add the switch enable_java_udf to control the creation and use of java_udf. (#26213) 2023-11-02 19:33:17 +08:00
谢健
c46fa339d0 [feature](Nereids): make count() avaliable for all data type (#26180) 
make count() avaliable for all data type
 2023-11-02 17:20:24 +08:00
Gabriel
6828250207 [profile](refactor) Fix invalid shuffle profile (#26298) 2023-11-02 17:07:05 +08:00
Gabriel
dd8bcc831c [keyword](decimalv2) Add DecimalV2 keyword (#26283) 2023-11-02 16:27:12 +08:00
Xinyi Zou
f1362e0d1e [fix](planner) Fix sample partition table (#25912) 
In the past, two conditions needed to be met when sampling a partitioned table: 1. Data is evenly distributed between partitions; 2. Data is evenly distributed between buckets. Finally, the number of sampled rows in each partition and each bucket is the same.

Now, sampling will be proportional to the number of partitioned and bucketed rows.
 2023-11-02 15:52:48 +08:00
walter
3e9e8becc6 [fix](regression) Add auto-expired prefix for backup/restore cases (#26272) 2023-11-02 15:21:43 +08:00
Pxl
1f9fdac148 [Chore](case) adjust some mv regression test case (#26279) 
adjust some mv regression test case
 2023-11-02 14:17:24 +08:00
Pxl
3004dbbc78 [Bug](materialized-view) SelectMaterializedIndexWithAggregate do not … (#26192) 
SelectMaterializedIndexWithAggregate do not change plan when match base index
 2023-11-02 14:12:22 +08:00
Qi Chen
3ce1bdae84 [Feature](Nereids) Add explain tree functionality. (#25446) 
Add explain tree functionality which contains fragment ids for profile analysis.
```
mysql> explain tree select sum(l_orderkey) from lineitem, orders where l_orderkey = o_orderkey;
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Explain String(Nereids Planner)                                                                                                                                                                                   |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| [414]:[414: ResultSink]||[Fragment: 0]||VRESULT SINK||   MYSQL_PROTOCAL||                                                                                                                                         |
| --[414]:[414: VAGGREGATE (merge finalize)]||[Fragment: 0]||cardinality=1||                                                                                                                                        |
| ----[411]:[411: VEXCHANGE]||[Fragment: 0]||offset: 0||                                                                                                                                                            |
| ------[411]:[411: DataStreamSink]||[Fragment: 1]||STREAM DATA SINK||  EXCHANGE ID: 411||  UNPARTITIONED                                                                                                           |
| --------[408]:[408: VAGGREGATE (update serialize)]||[Fragment: 1]||cardinality=1||                                                                                                                                |
| ----------[402]:[402: VHASH JOIN]||[Fragment: 1]||join op: INNER JOIN(PARTITIONED)[]||cardinality=593,132,346||                                                                                                   |
| ------------[392]:[392: VEXCHANGE]||[Fragment: 1]||offset: 0||                                                                                                                                                    |
| --------------[392]:[392: DataStreamSink]||[Fragment: 2]||STREAM DATA SINK||  EXCHANGE ID: 392||  HASH_PARTITIONED                                                                                                |
| ----------------[386]:[386: VHIVE_SCAN_NODE]||[Fragment: 2]||table: lineitem||inputSplitNum=144, totalFileSize=16632158739, scanRanges=144||partition=1/1||cardinality=600037902, numNodes=1||pushdown agg=NONE|| |
| ------------[399]:[399: VEXCHANGE]||[Fragment: 1]||offset: 0||                                                                                                                                                    |
| --------------[399]:[399: DataStreamSink]||[Fragment: 3]||STREAM DATA SINK||  EXCHANGE ID: 399||  HASH_PARTITIONED                                                                                                |
| ----------------[393]:[393: VHIVE_SCAN_NODE]||[Fragment: 3]||table: orders||inputSplitNum=48, totalFileSize=3984213353, scanRanges=48||partition=1/1||cardinality=150000000, numNodes=1||pushdown agg=NONE||      |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
12 rows in set (0.04 sec)
```
 2023-11-02 14:06:24 +08:00
zhangdong
fb622800aa [fix](mtmv)Compatible old mtmv metadata (#26193) 
make sure user can upgrade from v2.0 to master
 2023-11-02 12:04:04 +08:00