Commit Graph

5050 Commits

Author SHA1 Message Date
074323f112 [enhancement](Nereids): rearrange Limit optimization rules (#26378) 2023-11-03 17:13:44 +08:00
265c83d912 [fix](Planner): don't push down isNull predicate into view (#26288)
We can't push down `isNull` predicate into view
2023-11-03 16:30:12 +08:00
a4637e4462 [fix](fe) Add/remove frontends after updating BEBJE (#26313)
To ensure exception safety, adding or removing frontends in memory
should execute after removing the electable nodes (might throw
exceptions)
2023-11-03 14:33:06 +08:00
74fd5da420 [fix](Nereids) time extract function constant folding core (#26292)
some time extract function changed return type in the previous PR #18369
but it is not change FE constant folding function signature.
This is let them have same signature to avoid BE core.
2023-11-03 01:24:12 -05:00
a89477e8b5 [fix](Nereids) could not run query with repeat node in cte (#26330)
ExpressionDeepCopier not process VirtualReference, so we generate inline
plan with mistake.
2023-11-03 01:24:01 -05:00
0f24500ff8 [fix](Nereids) RewriteCteChildren not work with cost based rewritter (#26326)
we use a map to record rewrite cte children result to avoid rewrite
twice in cost based rewritter. However, we record cte outer and
inner in one map, and use null as outer result's key, use cte id as
inner result's key. This is wrong, because every anchor has an outer,
and we could only record one outer. So when we use the cache in cost
based rewritter, we get wrong outer plan from the cache. Then the error
will be thrown as below:

```
Caused by: java.lang.IllegalArgumentException: Stats for CTE: CTEId#1 not found
    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:143) ~[guava-32.1.2-jre.jar:?]
    at org.apache.doris.nereids.stats.StatsCalculator.visitLogicalCTEConsumer(StatsCalculator.java:1049) ~[classes/:?]
    at org.apache.doris.nereids.stats.StatsCalculator.visitLogicalCTEConsumer(StatsCalculator.java:147) ~[classes/:?]
    at org.apache.doris.nereids.trees.plans.logical.LogicalCTEConsumer.accept(LogicalCTEConsumer.java:111) ~[classes/:?]
    at org.apache.doris.nereids.stats.StatsCalculator.estimate(StatsCalculator.java:222) ~[classes/:?]
    at org.apache.doris.nereids.stats.StatsCalculator.estimate(StatsCalculator.java:200) ~[classes/:?]
    at org.apache.doris.nereids.jobs.cascades.DeriveStatsJob.execute(DeriveStatsJob.java:108) ~[classes/:?]
    at org.apache.doris.nereids.jobs.scheduler.SimpleJobScheduler.executeJobPool(SimpleJobScheduler.java:39) ~[classes/:?]
    at org.apache.doris.nereids.jobs.executor.Optimizer.execute(Optimizer.java:51) ~[classes/:?]
    at org.apache.doris.nereids.jobs.rewrite.CostBasedRewriteJob.getCost(CostBasedRewriteJob.java:98) ~[classes/:?]
    at org.apache.doris.nereids.jobs.rewrite.CostBasedRewriteJob.execute(CostBasedRewriteJob.java:64) ~[classes/:?]
    at org.apache.doris.nereids.jobs.executor.AbstractBatchJobExecutor.execute(AbstractBatchJobExecutor.java:119) ~[classes/:?]
    at org.apache.doris.nereids.rules.rewrite.RewriteCteChildren.visit(RewriteCteChildren.java:72) ~[classes/:?]
    at org.apache.doris.nereids.rules.rewrite.RewriteCteChildren.visit(RewriteCteChildren.java:56) ~[classes/:?]
    at org.apache.doris.nereids.trees.plans.visitor.PlanVisitor.visitLogicalSink(PlanVisitor.java:118) ~[classes/:?]
    at org.apache.doris.nereids.trees.plans.visitor.SinkVisitor.visitLogicalResultSink(SinkVisitor.java:72) ~[classes/:?]
    at org.apache.doris.nereids.trees.plans.logical.LogicalResultSink.accept(LogicalResultSink.java:58) ~[classes/:?]
    at org.apache.doris.nereids.rules.rewrite.RewriteCteChildren.visitLogicalCTEAnchor(RewriteCteChildren.java:86) ~[classes/:?]
    at org.apache.doris.nereids.rules.rewrite.RewriteCteChildren.visitLogicalCTEAnchor(RewriteCteChildren.java:56) ~[classes/:?]
    at org.apache.doris.nereids.trees.plans.logical.LogicalCTEAnchor.accept(LogicalCTEAnchor.java:60) ~[classes/:?]
    at org.apache.doris.nereids.rules.rewrite.RewriteCteChildren.visitLogicalCTEAnchor(RewriteCteChildren.java:86) ~[classes/:?]
    at org.apache.doris.nereids.rules.rewrite.RewriteCteChildren.visitLogicalCTEAnchor(RewriteCteChildren.java:56) ~[classes/:?]
    at org.apache.doris.nereids.trees.plans.logical.LogicalCTEAnchor.accept(LogicalCTEAnchor.java:60) ~[classes/:?]
    at org.apache.doris.nereids.rules.rewrite.RewriteCteChildren.rewriteRoot(RewriteCteChildren.java:67) ~[classes/:?]
    at org.apache.doris.nereids.jobs.rewrite.CustomRewriteJob.execute(CustomRewriteJob.java:58) ~[classes/:?]
    at org.apache.doris.nereids.jobs.executor.AbstractBatchJobExecutor.execute(AbstractBatchJobExecutor.java:119) ~[classes/:?]
    at org.apache.doris.nereids.NereidsPlanner.rewrite(NereidsPlanner.java:275) ~[classes/:?]
    at org.apache.doris.nereids.NereidsPlanner.plan(NereidsPlanner.java:218) ~[classes/:?]
    at org.apache.doris.nereids.NereidsPlanner.plan(NereidsPlanner.java:118) ~[classes/:?]
    at org.apache.doris.nereids.trees.plans.commands.ExplainCommand.run(ExplainCommand.java:81) ~[classes/:?]
    at org.apache.doris.qe.StmtExecutor.executeByNereids(StmtExecutor.java:550) ~[classes/:?]
```
2023-11-03 01:12:31 -05:00
6892fc99f8 [fix](Nereids) only search internal funcftion when dbName is empty (#26296)
if call function with database name. we should only search UDF
2023-11-03 01:01:25 -05:00
9019a71770 [Fix](multi-catalog) Filter _temporary dir which is generated by spark. (#26194)
Sometimes there are some temporary files which are generated by some spark jobs, we need filter these files.

```
errCode = 2, detailMessage = (xxx.xxx.xxx.xxx.xxx)
[CANCELLED][INTERNAL_ERROR]failed to init reader for file hdfs://xxx/user/hive/warehouse/xxx.db/xxx/_temporary/0/_temporary/attempt_202311010952379045223848154629697_0013_m_000442_3355/part-00442-3f97ac18-0583-4d2c-8995-8d21ceda56ec-c000.snappy.orc, 
err: [INTERNAL_ERROR]Init OrcReader failed. reason = Failed to parse the postscript from hdfs://xxx/user/hive/warehouse/xxx.db/xxx/_temporary/0/_temporary/attempt_202311010952379045223848154629697_0013_m_000442_3355/part-00442-3f97ac18-0583-4d2c-8995-8d21ceda56ec-c000.snappy.orc   
0.  @ 0x000000000bdbd89f
   doris::Status doris::Status::Error<6, true, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>(std::basic_string_view<char, std::char_traits<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)  /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:173  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  1.  @ 0x000000000eccecfe
   doris::vectorized::OrcReader::_create_file_reader()  /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  2.  @ 0x000000000ecceff9
   doris::vectorized::OrcReader::init_reader(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const*, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::variant<doris::ColumnValueRange<(doris::PrimitiveType)3>, doris::ColumnValueRange<(doris::PrimitiveType)4>, doris::ColumnValueRange<(doris::PrimitiveType)5>, doris::ColumnValueRange<(doris::PrimitiveType)6>, doris::ColumnValueRange<(doris::PrimitiveType)7>, doris::ColumnValueRange<(doris::PrimitiveType)15>, doris::ColumnValueRange<(doris::PrimitiveType)10>, doris::ColumnValueRange<(doris::PrimitiveType)23>, doris::ColumnValueRange<(doris::PrimitiveType)11>, doris::ColumnValueRange<(doris::PrimitiveType)25>, doris::ColumnValueRange<(doris::PrimitiveType)12>, doris::ColumnValueRange<(doris::PrimitiveType)26>, doris::ColumnValueRange<(doris::PrimitiveType)20>, doris::ColumnValueRange<(doris::PrimitiveType)2>, doris::ColumnValueRange<(doris::PrimitiveType)19>, doris::ColumnValueRange<(doris::PrimitiveType)28>, doris::ColumnValueRange<(doris::PrimitiveType)29>, doris::ColumnValueRange<(doris::PrimitiveType)30> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::variant<doris::ColumnValueRange<(doris::PrimitiveType)3>, doris::ColumnValueRange<(doris::PrimitiveType)4>, doris::ColumnValueRange<(doris::PrimitiveType)5>, doris::ColumnValueRange<(doris::PrimitiveType)6>, doris::ColumnValueRange<(doris::PrimitiveType)7>, doris::ColumnValueRange<(doris::PrimitiveType)15>, doris::ColumnValueRange<(doris::PrimitiveType)10>, doris::ColumnValueRange<(doris::PrimitiveType)23>, doris::ColumnValueRange<(doris::PrimitiveType)11>, doris::ColumnValueRange<(doris::PrimitiveType)25>, doris::ColumnValueRange<(doris::PrimitiveType)12>, doris::ColumnValueRange<(doris::PrimitiveType)26>, doris::ColumnValueRange<(doris::PrimitiveType)20>, doris::ColumnValueRange<(doris::PrimitiveType)2>, doris::ColumnValueRange<(doris::PrimitiveType)19>, doris::ColumnValueRange<(doris::PrimitiveType)28>, doris::ColumnValueRange<(doris::PrimitiveType)29>, doris::ColumnValueRange<(doris::PrimitiveType)30> > > > >*, std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std::shared_ptr<doris::vectorized::VExprContext> > > const&, bool, doris::TupleDescriptor const*, doris::RowDescriptor const*, std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std::shared_ptr<doris::vectorized::VExprContext> > > const*, std::unordered_map<int, std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std::shared_ptr<doris::vectorized::VExprContext> > >, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std::shared_ptr<doris::vectorized::VExprContext> > > > > > const*)  /root/doris-master/be/src/common/status.h:448  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  3.  @ 0x000000000ff43da0
   doris::vectorized::VFileScanner::_get_next_reader()  /root/doris-master/be/src/vec/exec/scan/vfile_scanner.cpp:800  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  4.  @ 0x000000000ff4256f
   doris::vectorized::VFileScanner::_get_block_impl(doris::RuntimeState*, doris::vectorized::Block*, bool*)  /root/doris-master/be/src/common/status.h:448  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  5.  @ 0x000000000ffdb1aa
   doris::vectorized::VScanner::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*)  /root/doris-master/be/src/vec/exec/scan/vscanner.cpp:0  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  6.  @ 0x000000000ff3b379
   doris::vectorized::ScannerScheduler::_scanner_scan(doris::vectorized::ScannerScheduler*, doris::vectorized::ScannerContext*, std::shared_ptr<doris::vectorized::VScanner>)  /root/doris-master/be/src/common/status.h:346  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  7.  @ 0x000000000ff3ca11  std::_Function_handler<void (), doris::vectorized::ScannerScheduler::_schedule_scanners(doris::vectorized::ScannerContext*)::$_1::operator()() const::{lambda()#4}>::_M_invoke(std::_Any_data const&)  /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  8.  @ 0x000000000c986608
   doris::ThreadPool::dispatch_thread()  /root/doris-master/be/src/util/threadpool.cpp:0  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  9.  @ 0x000000000c97c2ac
   doris::Thread::supervise_thread(void*)  /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  10. @ 0x0000000000007ea5  start_thread  in /usr/lib64/libpthread-2.17.so  11. @ 0x00000000000feb0d  __clone  in /usr/lib64/libc-2.17.so    0.  @ 0x000000000c647366
   doris::Status doris::Status::Error<6, true, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(std::basic_string_view<char, std::char_traits<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&)  /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:173  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  1.  @ 0x000000000ff42dc3
   doris::vectorized::VFileScanner::_get_next_reader()  /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  2.  @ 0x000000000ff4256f
   doris::vectorized::VFileScanner::_get_block_impl(doris::RuntimeState*, doris::vectorized::Block*, bool*)  /root/doris-master/be/src/common/status.h:448  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  3.  @ 0x000000000ffdb1aa
   doris::vectorized::VScanner::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*)  /root/doris-master/be/src/vec/exec/scan/vscanner.cpp:0  in /hdpData10/yarn/localcache/usercache/hive/appcache/application_1696938614818_1832823/filecache/13/doris-be-360-master.tar.gz/apache-doris-be-360-master-bin-x86_64/lib/doris_be  4.  @ 0x000000000ff3b379
   doris::vectorized::ScannerScheduler::_scanner_scan(doris::vectorized::ScannerScheduler*, doris::vectorized::ScannerContext*, std::shared_ptr<doris::vectorized::VScanner>)  /root/doris-master/be/src/common/status.h:346  in /hdpData10/yarn/localcache/usercache/hive/appcache/a…
```
Co-authored-by: wangxiangyu <wangxiangyu@360shuke.com>
2023-11-03 13:42:29 +08:00
b19f275714 [improvement](insert) refactor group commit insert into (#25795) 2023-11-03 12:02:40 +08:00
61edb9e4d8 [feature](mtmv)(2)Implementing mtmv using antlr (#26102)
Implementing mtmv using antlr,

No specific business logic was implemented, therefore an exception was thrown during execution
```
throw new AnalysisException("current not support.");
```
2023-11-03 11:57:00 +08:00
93a934e775 [Improve](map) support map cast with map literal and implicate nested scala cast (#26126) 2023-11-02 09:56:42 -05:00
e307884e57 [enhancement](Nereids): GroupPlan don't generate ObjectId (#26315) 2023-11-02 21:35:48 +08:00
d9621404b3 [improvement](fe) Add more log about add/remove frontends (#26306) 2023-11-02 20:19:51 +08:00
a5ef90dacc [enhancement](recover) support skipping missing version in select by session variable (#25654) 2023-11-02 20:01:51 +08:00
89cf828f03 [enhancement](udf) add the switch enable_java_udf to control the creation and use of java_udf. (#26213) 2023-11-02 19:33:17 +08:00
c46fa339d0 [feature](Nereids): make count() avaliable for all data type (#26180)
make count() avaliable for all data type
2023-11-02 17:20:24 +08:00
f1362e0d1e [fix](planner) Fix sample partition table (#25912)
In the past, two conditions needed to be met when sampling a partitioned table: 1. Data is evenly distributed between partitions; 2. Data is evenly distributed between buckets. Finally, the number of sampled rows in each partition and each bucket is the same.

Now, sampling will be proportional to the number of partitioned and bucketed rows.
2023-11-02 15:52:48 +08:00
Pxl
3004dbbc78 [Bug](materialized-view) SelectMaterializedIndexWithAggregate do not … (#26192)
SelectMaterializedIndexWithAggregate do not change plan when match base index
2023-11-02 14:12:22 +08:00
3ce1bdae84 [Feature](Nereids) Add explain tree functionality. (#25446)
Add explain tree functionality which contains fragment ids for profile analysis.
```
mysql> explain tree select sum(l_orderkey) from lineitem, orders where l_orderkey = o_orderkey;
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Explain String(Nereids Planner)                                                                                                                                                                                   |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| [414]:[414: ResultSink]||[Fragment: 0]||VRESULT SINK||   MYSQL_PROTOCAL||                                                                                                                                         |
| --[414]:[414: VAGGREGATE (merge finalize)]||[Fragment: 0]||cardinality=1||                                                                                                                                        |
| ----[411]:[411: VEXCHANGE]||[Fragment: 0]||offset: 0||                                                                                                                                                            |
| ------[411]:[411: DataStreamSink]||[Fragment: 1]||STREAM DATA SINK||  EXCHANGE ID: 411||  UNPARTITIONED                                                                                                           |
| --------[408]:[408: VAGGREGATE (update serialize)]||[Fragment: 1]||cardinality=1||                                                                                                                                |
| ----------[402]:[402: VHASH JOIN]||[Fragment: 1]||join op: INNER JOIN(PARTITIONED)[]||cardinality=593,132,346||                                                                                                   |
| ------------[392]:[392: VEXCHANGE]||[Fragment: 1]||offset: 0||                                                                                                                                                    |
| --------------[392]:[392: DataStreamSink]||[Fragment: 2]||STREAM DATA SINK||  EXCHANGE ID: 392||  HASH_PARTITIONED                                                                                                |
| ----------------[386]:[386: VHIVE_SCAN_NODE]||[Fragment: 2]||table: lineitem||inputSplitNum=144, totalFileSize=16632158739, scanRanges=144||partition=1/1||cardinality=600037902, numNodes=1||pushdown agg=NONE|| |
| ------------[399]:[399: VEXCHANGE]||[Fragment: 1]||offset: 0||                                                                                                                                                    |
| --------------[399]:[399: DataStreamSink]||[Fragment: 3]||STREAM DATA SINK||  EXCHANGE ID: 399||  HASH_PARTITIONED                                                                                                |
| ----------------[393]:[393: VHIVE_SCAN_NODE]||[Fragment: 3]||table: orders||inputSplitNum=48, totalFileSize=3984213353, scanRanges=48||partition=1/1||cardinality=150000000, numNodes=1||pushdown agg=NONE||      |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
12 rows in set (0.04 sec)
```
2023-11-02 14:06:24 +08:00
fb622800aa [fix](mtmv)Compatible old mtmv metadata (#26193)
make sure user can upgrade from v2.0 to master
2023-11-02 12:04:04 +08:00
80f654ec2a [Fix](statistics)Fix analyze min max sql syntax error. #26240 2023-11-02 09:22:32 +08:00
6010be88bd [enhancement](Nereids): optimize GroupExpressionMatching (#26196) 2023-11-01 19:05:08 +08:00
502f5778f4 [enhancement](Nereids): set maxJoinNumBushyTree = 8 (#26173) 2023-11-01 19:04:49 +08:00
1c2831e06f [fix](spill) disable spill of sort and agg for now to avoid diisk overflow (#26209) 2023-11-01 17:20:26 +08:00
683832230c [Performance](Nereids): pass ConnectContext to avoid ThreadLocal.get() (#26165) 2023-11-01 16:16:42 +08:00
1770224322 [fix](planner) Fix decimal precision and scale wrong when create table like (#25802)
Use field datatype such as decimal(10, 0) to create table like. Because the scale is 0, the precision and scale will lost when create table like done. this will fix the bug.

**Before fix, create table with following SQL**:
CREATE TABLE IF NOT EXISTS db_test.table_test
(
    `name` varchar COMMENT "1m size",
    `id` SMALLINT COMMENT "[-32768, 32767]",
    `timestamp0` decimal null comment "c0",
    `timestamp1` decimal(38, 0) null comment "c1"
)
DISTRIBUTED BY HASH(`id`) BUCKETS 1
PROPERTIES ('replication_num' = '1');

**and Then run**
CREATE TABLE db_test.table_test_like LIKE db_test.table_test
SHOW CREATE TABLE db_test.table_test_like;

the field `timestamp1` will be decimal(9, 0), it's wrong. this will fix it.
2023-11-01 03:11:43 -05:00
5efc6c4233 [fix](auth)Fix can not insert overwrite who have load priv (#25711)
Fix can not insert overwrite who have load priv
2023-11-01 15:21:16 +08:00
HB
6de1cc7a0a [fix](udf)java udf does not support overloaded evaluate method (#22681)
Co-authored-by: morningman <morningman@163.com>
2023-11-01 15:05:37 +08:00
268c69971d [fix](stats) Store max/min by base64 2023-11-01 14:31:35 +08:00
eaed0dea5e [refactor](executor)Add % suffix for cpu_hard_limit #26174 2023-11-01 10:27:48 +08:00
18dabe7386 [fix](compile) fe compile failed when generate doc and FE UT failed (#26164)
1. FE could not compile because below error. Intro by PR #25933
```
[INFO] --- exec:3.1.0:java (doc) @ fe-core ---
...
Failed to generate doc for ignoreRuntimeFilterIds
```

2. fix UT bugs intro by below PRs
> - #25951
> - #26031

3. because fe could not compile, FE UT CI do not work well. So, some UT failed be introduced by the PRs merged after PR #25933 merged. So this PR revert them to fix FE UT

> - Revert "[Bug](materialized-view) SelectMaterializedIndexWithAggregate do not change plan > when match ba… (#26145)"
> This reverts commit 8d7abf60f94d2d1208b71e96b9290ea02122b8d8.

> - Revert "[enhancement](Nereids): optimize GroupExpressionMatching (#26130)"
> This reverts commit 19122b55cd95af097b4ef7b6eb809f37db29765f.

> - Revert "[Performance](Nereids): optimize GroupExpressionMatching (#26084)"
> This reverts commit 0d956e90cf920039b8baa79c170a298be56a128d.
2023-10-31 20:50:44 -05:00
ac7678926f [improvement](tablet clone) add config for decommission tablet wait time (#26000) 2023-10-31 20:36:26 +08:00
08c78a1135 [Feature](Job)Support manual and refactor some execution logic (#26082)
Supports manually triggered JOBs and Tasks
Optimize JOB&TASK display logic
Refactor the executor to support context passing
2023-10-31 20:35:55 +08:00
b98744ae90 [Bug](iceberg)fix read partitioned iceberg without partition path (#25503)
Iceberg does not require partition values to exist on file paths, so we should get the partition value from `PartitionScanTask.partition`.
2023-10-31 18:09:53 +08:00
b137f03921 [feature](profile) add MaxActiveTime and add exectime in pipelineX (#26148) 2023-10-31 17:53:52 +08:00
Pxl
271b074f42 Bug](view) make forbiddenMVRewrite work on InlineViewRef (#26162)
make forbiddenMVRewrite work on InlineViewRef
2023-10-31 17:30:59 +08:00
6f316b117f [Fix](Checkpoint)AnalysisMgrV2 does not implement the corresponding read and write interface (#26161) 2023-10-31 17:24:35 +08:00
994db8b400 [fix](multi-catalog)add the FAQ for Aliyun DLF and add the fs.xx.impl check (#25594)
1. add the FAQ for Aliyun DLF,include the DLF configuration and Jindo SDK locations
2. add the fs.xx.impl check when load cache files
2023-10-31 17:09:52 +08:00
958c7fdfd9 [fix](planner)should keep at least one slot materialized in agg node (#26116) 2023-10-31 16:52:45 +08:00
a6e010f722 [opt](nereids) use rowcount to compute runtime filter size if ndv is not available (#26022) 2023-10-31 16:52:17 +08:00
a259ae9417 [fix](mtmv)add mtmvJobManager to DEPRECATED_MODULE_NAMES (#26137)
Inherited from:26041
2023-10-31 16:28:43 +08:00
19122b55cd [enhancement](Nereids): optimize GroupExpressionMatching (#26130)
1. pattern can't be SubTreePattern in CBO phase.
2. optimize getInputSlotExprId()
2023-10-31 15:47:13 +08:00
feefebe0da [chore](log opt) opt mysql load log (#26149) 2023-10-31 14:59:52 +08:00
ea1e8fa15d [Fix](Nereids) fix create table as select of view with unknowed length character type (#25471)
Problem:
when create table as select from a view with unknown length character type, be would return an error of inserting data failed

Example:
doris/regression-test/suites/ddl_p0/test_ctas.groovy

Reason & Solved:
BE can not derive varchar length automaticly so FE should tell BE to maximize the size of varchar type
2023-10-31 01:50:28 -05:00
0449a240f4 [Fix](from_unixtime) Keep consistent with MySQL & bug fix (#25966)
Bug fix: implicit convert from int32 -> int64 makes negative time stamp valid, so change signature to int64
Consistent: keep consistent with mysql.
2023-10-31 14:31:24 +08:00
Pxl
8d7abf60f9 [Bug](materialized-view) SelectMaterializedIndexWithAggregate do not change plan when match ba… (#26145)
* SelectMaterializedIndexWithAggregate do not change plan when match base index

* forbid table withh alias
2023-10-31 14:11:08 +08:00
78204f7c92 [Fix](statistics)Fix external couldn't analyze database bug (#26025) 2023-10-31 11:32:47 +08:00
b3f31f9e1a [enhancement](Nereids): speedup graphsimplifier (#26066)
1. fix some bugs in graphsimplifier
2. remove some time costed code
3. add shape check for graphsimplifier
2023-10-30 20:13:13 +08:00
2a74d9a8c8 [feature](mtmv)(1)remove old mtmv code (#26041)
remove old mtmv code,we will implement mtmv in a new way
2023-10-30 19:49:45 +08:00
f97c40d0e1 [fix](Nereids) fix plan shape of query64 unstable (#26012)
don't remove the physical plan after optimizing the plan in dphyper.
2023-10-30 19:49:28 +08:00