database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
19,775 Commits 1 Branch 0 Tags
226e01889c4e36a9ed0c7cd5273f4fa02a3bfe2f
Commit Graph

19775 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
amory
226e01889c [fix](array_apply) pick array apply fix (#39328) 
## Proposed changes
backport: https://github.com/apache/doris/pull/39105
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-14 18:52:29 +08:00
amory
78d6e318fb [fix](ip)pick ip rowstore (#39345) 
## Proposed changes
backport: https://github.com/apache/doris/pull/39258
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-14 18:51:58 +08:00
Mryange
b26af32934 [fix](function) fix error return type in corr(float32,float32) (#39251) (#39350) 
https://github.com/apache/doris/pull/39251
```
mysql [test11]>select corr(cast(x as float),cast(y as float)) from test_corr;
ERROR 1105 (HY000): errCode = 2, detailMessage = (127.0.0.1)[INTERNAL_ERROR]column_type not match data_types in agg node, column_type=Nullable(Float64), data_types=Nullable(Float32),column name=

```

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-14 18:47:14 +08:00
Mryange
a9692a305e [fix](function)timediff with now function causes a error signature (… (#39349) 
…#39322)
https://github.com/apache/doris/pull/39322
## Proposed changes

```
mysql [(none)]>select round(timediff(now(),'2024-08-15')/60/60,2);
ERROR 1105 (HY000): errCode = 2, detailMessage = argument 1 requires datetimev2 type, however 'now()' is of datetime type
```
The reason is that the function parameter types were modified in
expectedInputTypes, which led to no match being found. The code here is
from a long time ago. Because the precision of datetimev2 could not be
deduced in the past, a separate implementation was made here. This code
can be safely deleted.


<!--Describe your changes.-->

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-14 18:36:14 +08:00
feiniaofeiafei
701e23b65b [Fix](nereids) fix condition function partition prune (#39298) (#39332) 
cherry-pick #39298 to branch-2.1
 2024-08-14 18:32:50 +08:00
Tiewei Fang
d2709f8600 [fix](test) fix test_numbers case (#39303) 
bp: #38687

we use `order_qt` rather than `qt` to promise the order of results.
 2024-08-13 22:26:39 +08:00
Tiewei Fang
187461e2fd [Fix](Export) Export delete multiple times when specify the delete_existing_files property () (#39304) 
bp: #38400

When the `Export` statement specifies the `delete_existing_files`
property, each `Outfile` statement generated by the `Export` will carry
this property. This causes each `Outfile` statement to delete existing
files, so only the result of the last Outfile statement will be
retained.

So, we add a rpc method which can delete existing files for `Export`
statement and the `Outfile` statements generated by the `Export` will
not carry `delete_existing_files` property any more.

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-13 22:26:02 +08:00
Pxl
6f1d9812bb [Bug](brpc) fix sync_filter_size/apply_filterv2 has wrong closure (#39299) 
pick from #39155
 2024-08-13 19:01:22 +08:00
zfr95
aeffb18ba6 [test](mtmv)Add the cases of upgrade and downgrade of mtmv (#39289) 
pick from #38868 

## Proposed changes

[test](mtmv)Add the cases of upgrade and downgrade of mtmv
 2024-08-13 17:46:26 +08:00
lihangyu
677435cef8 [Pick](Branch-2.1) pick json reader fix and support specify $. as column (#39271) 
#39206
#38213
 2024-08-13 17:44:45 +08:00
zhangstar333
7e7729c4b0 [cherry-pick](branch-21) fix partition-topn calculate partition input rows have error (#39100) (#39281) 
## Proposed changes

cherry-pick from master: #39100 

<!--Describe your changes.-->
 2024-08-13 17:42:29 +08:00
Luwei
c9949f24e5 [fix](compaction) fix the longest continuous rowsets cannot be selected when missing rowsets (#38728) (#39262) 
pick master #38728
 2024-08-13 17:41:11 +08:00
camby
b976cbf14d [opt](log) avoid lots of json parse error logs (#39190) (#39246) 
pick https://github.com/apache/doris/pull/39190 to branch-2.1
 2024-08-13 17:01:10 +08:00
Pxl
33220109f7 [Bug](materialized-view) fix analyze where clause failed on mv (#39061) (#39209) 
## Proposed changes
pick from #39061
fix analyze where clause failed on mv
do not analyze slot after replaceSlot to avoid duplicate columns in desc
 2024-08-13 16:08:20 +08:00
htyoung
9fb103e979 [opt](fe) Optimize calculate load job num metric in FE (#39267) 
cherry pick from https://github.com/apache/doris/pull/31952
https://github.com/apache/doris/pull/34020
Co-authored-by: Lei Zhang <27994433+SWJTU-ZhangLei@users.noreply.github.com>
 2024-08-13 15:42:15 +08:00
qiye
000ea20562 [fix](inverted index)Add exception check when write bkd index (#39248) (#39277) 
bp #39248
 2024-08-13 15:14:16 +08:00
qiye
43cc8d648d [fix](ES Catalog)Check isArray before parse json to array (#39104) (#39273) 
## Proposed changes

bp #39104
 2024-08-13 15:13:40 +08:00
walter
8f9a858452 [fix](suites) fix backup cancelled cases #39204 (#39274) 
cherry pick from #39204
 2024-08-13 14:08:58 +08:00
Mryange
0309f0f6b0 [fix](pipeline) LocalExchangeSource was closing without opening (#39237) (#39266) 
## Proposed changes
https://github.com/apache/doris/pull/39237
which led to accessing uninitialized data. Moved the profile
initialization into the init method.
```
/mnt/disk2/yanxuecheng/doris/be/src/pipeline/local_exchange/local_exchanger.cpp💯9: runtime error: member call on null pointer of type 'doris::RuntimeProfile::Counter'
    #0 0x563f0fb8b6c9 in doris::pipeline::Exchanger<std::shared_ptr<doris::pipeline::BlockWrapper>>::_dequeue_data(doris::pipeline::LocalExchangeSourceLocalState&, std::shared_ptr<doris::pipeline::BlockWrapper>&, bool*, doris::vectorized::Block*, int) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/local_exchange/local_exchanger.cpp💯9
    #1 0x563f0fb78a45 in doris::pipeline::LocalMergeSortExchanger::finalize(doris::pipeline::LocalExchangeSourceLocalState&) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/local_exchange/local_exchanger.cpp:325:16
    #2 0x563f09610366 in doris::pipeline::LocalExchangeSharedState::sub_running_source_operators(doris::pipeline::LocalExchangeSourceLocalState&) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/dependency.cpp:196:20
    #3 0x563f0fb63709 in doris::pipeline::LocalExchangeSourceLocalState::close(doris::RuntimeState*) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/local_exchange/local_exchange_source_operator.cpp:59:24
    #4 0x563f09728e80 in doris::pipeline::OperatorXBase::close(doris::RuntimeState*) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/exec/operator.cpp:245:28
    #5 0x563f0fd3fb58 in doris::pipeline::PipelineTask::close(doris::Status) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/pipeline_task.cpp:459:28
    #6 0x563f0fdb5315 in doris::pipeline::_close_task(doris::pipeline::PipelineTask*, doris::Status) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/task_scheduler.cpp:91:27
    #7 0x563f0fdb6573 in doris::pipeline::TaskScheduler::_do_work(unsigned long) /mnt/disk2/yanxuecheng/doris/be/src/pipeline/task_scheduler.cpp:125:13
    #8 0x563f0fdb9d6a in doris::pipeline::TaskScheduler::start()::$_0::operator()() const /mnt/disk2/yanxuecheng/doris/be/src/pipeline/task_scheduler.cpp:64:9
    #9 0x563f0fdb9cee in void std::__invoke_impl<void, doris::pipeline::TaskScheduler::start()::$_0&>(std::__invoke_other, doris::pipeline::TaskScheduler::start()::$_0&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14
    #10 0x563f0fdb9c4e in std::enable_if<is_invocable_r_v<void, doris::pipeline::TaskScheduler::start()::$_0&>, void>::type std::__invoke_r<void, doris::pipeline::TaskScheduler::start()::$_0&>(doris::pipeline::TaskScheduler::start()::$_0&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2
    #11 0x563f0fdb99d5 in std::_Function_handler<void (), doris::pipeline::TaskScheduler::start()::$_0>::_M_invoke(std::_Any_data const&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9
    #12 0x563ebdddc8cf in std::function<void ()>::operator()() const /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9
    #13 0x563ec4bd6db4 in doris::FunctionRunnable::run() /mnt/disk2/yanxuecheng/doris/be/src/util/threadpool.cpp:48:27
    #14 0x563ec4bbc1b5 in doris::ThreadPool::dispatch_thread() /mnt/disk2/yanxuecheng/doris/be/src/util/threadpool.cpp:543:24
    #15 0x563ec4bf9a53 in void std::__invoke_impl<void, void (doris::ThreadPool::*&)(), doris::ThreadPool*&>(std::__invoke_memfun_deref, void (doris::ThreadPool::*&)(), doris::ThreadPool*&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:74:14
    #16 0x563ec4bf9858 in std::__invoke_result<void (doris::ThreadPool::*&)(), doris::ThreadPool*&>::type std::__invoke<void (doris::ThreadPool::*&)(), doris::ThreadPool*&>(void (doris::ThreadPool::*&)(), doris::ThreadPool*&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:96:14
    #17 0x563ec4bf9790 in void std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>::__call<void, 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/functional:506:11
    #18 0x563ec4bf9585 in void std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>::operator()<void>() /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/functional:591:17
    #19 0x563ec4bf947e in void std::__invoke_impl<void, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14
    #20 0x563ec4bf93be in std::enable_if<is_invocable_r_v<void, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&>(std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:111:2
    #21 0x563ec4bf8e55 in std::_Function_handler<void (), std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>>::_M_invoke(std::_Any_data const&) /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290:9
    #22 0x563ebdddc8cf in std::function<void ()>::operator()() const /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591:9
    #23 0x563ec4b78d91 in doris::Thread::supervise_thread(void*) /mnt/disk2/yanxuecheng/doris/be/src/util/thread.cpp:498:5
    #24 0x563ebdb2fe0a in asan_thread_start(void*) crtstuff.c
    #25 0x7feeac9e21c9 in start_thread (/lib64/libpthread.so.0+0x81c9) (BuildId: 823fccea3475e5870a4167dfe47df20e53222db0)
    #26 0x7feead3d1e72 in clone (/lib64/libc.so.6+0x39e72) (BuildId: ec3d7025354f1f1985831ff08ef0eb3b50aefbce)

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /mnt/disk2/yanxuecheng/doris/be/src/pipeline/local_exchange/local_exchanger.cpp💯9 in 
*** Query id: ea174401bc134452-bd8a35522726a96a ***
*** is nereids: 1 ***
*** tablet id: 0 ***
*** Aborted at 1723455006 (unix time) try "date -d @1723455006" if you are using GNU date ***
*** Current BE git commitID: c47399cc0d ***
*** SIGSEGV address not mapped to object (@0x0) received by PID 3055435 (TID 3060341 OR 0x7fe320acf700) from PID 0; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /mnt/disk2/yanxuecheng/doris/be/src/common/signal_handler.h:421
 1# 0x00007FEEAD3E6B50 in /lib64/libc.so.6
 2# doris::pipeline::Exchanger<std::shared_ptr<doris::pipeline::BlockWrapper> >::_dequeue_data(doris::pipeline::LocalExchangeSourceLocalState&, std::shared_ptr<doris::pipeline::BlockWrapper>&, bool*, doris::vectorized::Block*, int) at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/local_exchange/local_exchanger.cpp:100
 3# doris::pipeline::LocalMergeSortExchanger::finalize(doris::pipeline::LocalExchangeSourceLocalState&) at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/local_exchange/local_exchanger.cpp:325
 4# doris::pipeline::LocalExchangeSharedState::sub_running_source_operators(doris::pipeline::LocalExchangeSourceLocalState&) at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/dependency.cpp:196
 5# doris::pipeline::LocalExchangeSourceLocalState::close(doris::RuntimeState*) in /mnt/disk2/yanxuecheng/doris/output/be/lib/doris_be
 6# doris::pipeline::OperatorXBase::close(doris::RuntimeState*) at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/exec/operator.cpp:245
 7# doris::pipeline::PipelineTask::close(doris::Status) at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/pipeline_task.cpp:459
 8# doris::pipeline::_close_task(doris::pipeline::PipelineTask*, doris::Status) at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/task_scheduler.cpp:91
 9# doris::pipeline::TaskScheduler::_do_work(unsigned long) at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/task_scheduler.cpp:125
10# doris::pipeline::TaskScheduler::start()::$_0::operator()() const at /mnt/disk2/yanxuecheng/doris/be/src/pipeline/task_scheduler.cpp:64
11# void std::__invoke_impl<void, doris::pipeline::TaskScheduler::start()::$_0&>(std::__invoke_other, doris::pipeline::TaskScheduler::start()::$_0&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61
12# std::enable_if<is_invocable_r_v<void, doris::pipeline::TaskScheduler::start()::$_0&>, void>::type std::__invoke_r<void, doris::pipeline::TaskScheduler::start()::$_0&>(doris::pipeline::TaskScheduler::start()::$_0&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:117
13# std::_Function_handler<void (), doris::pipeline::TaskScheduler::start()::$_0>::_M_invoke(std::_Any_data const&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290
14# std::function<void ()>::operator()() const at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591
15# doris::FunctionRunnable::run() at /mnt/disk2/yanxuecheng/doris/be/src/util/threadpool.cpp:48
16# doris::ThreadPool::dispatch_thread() at /mnt/disk2/yanxuecheng/doris/be/src/util/threadpool.cpp:543
17# void std::__invoke_impl<void, void (doris::ThreadPool::*&)(), doris::ThreadPool*&>(std::__invoke_memfun_deref, void (doris::ThreadPool::*&)(), doris::ThreadPool*&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:74
18# std::__invoke_result<void (doris::ThreadPool::*&)(), doris::ThreadPool*&>::type std::__invoke<void (doris::ThreadPool::*&)(), doris::ThreadPool*&>(void (doris::ThreadPool::*&)(), doris::ThreadPool*&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:96
19# void std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/functional:506
20# void std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>::operator()<, void>() at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/functional:591
21# void std::__invoke_impl<void, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61
22# std::enable_if<is_invocable_r_v<void, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&>(std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:117
23# std::_Function_handler<void (), std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()> >::_M_invoke(std::_Any_data const&) at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290
24# std::function<void ()>::operator()() const at /mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591
25# doris::Thread::supervise_thread(void*) at /mnt/disk2/yanxuecheng/doris/be/src/util/thread.cpp:498
26# asan_thread_start(void*) in /mnt/disk2/yanxuecheng/doris/output/be/lib/doris_be
27# start_thread in /lib64/libpthread.so.0
28# __clone in /lib64/libc.so.6

```

<!--Describe your changes.-->

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-13 12:30:35 +08:00
Pxl
8cc9830b26 [Bug](http-api) fix core dump on /api/reset_rpc_channel coz exec_env not initialized (#39207) 
pick from #38375
 2024-08-13 11:54:36 +08:00
Sun Chenyang
228f78b80d [fix] (nereids) fix Match Expreesion in filter estimation (#39050) (#39215) 
## Proposed changes

pick from master #39050
 2024-08-13 10:57:53 +08:00
Sun Chenyang
60eeec3754 [fix] (inverted index) Fix match function without inverted index (#38989) (#39220) 
## Proposed changes

pick from #38989
 2024-08-13 10:55:54 +08:00
Sun Chenyang
a6155a517d [fix] (topn) fix uncleared block in topn_next() (#39119) (#39224) 
## Proposed changes

pick from master #39119
 2024-08-13 10:34:17 +08:00
Sun Chenyang
beab6a81c1 [fix] (compaction) fix time series compaction policy (#39170) (#39228) 
## Proposed changes

pick from master #39170
 2024-08-13 10:33:04 +08:00
yujun
f5e896f6eb [improvement](balance) don't balance tablet which has unfinish alter job #39121 (#39202) 
cherry pick from #39121
 2024-08-13 09:33:26 +08:00
bobhan1
ee6f9d0e90 [branch-2.1](regression) Fix case test_schema_change_unique_mow (#39173)(#39196) (#39213) 
## Proposed changes

picks https://github.com/apache/doris/pull/39173 and
https://github.com/apache/doris/pull/39196
 2024-08-12 19:04:49 +08:00
bobhan1
c2044409da [branch-2.1] Picks "[fix](delete) Fix delete stmt on MOW table doesn't use partial update in Nereids planner #38751" (#39214) 
## Proposed changes

picks https://github.com/apache/doris/pull/38751
 2024-08-12 17:02:48 +08:00
airborne12
01c39d73c2 [Refact](inverted index) refact no need read data opt logic #36969 (#39225) 
cherry pick from #36969
 2024-08-12 16:47:13 +08:00
LiBinfeng
0c39b88804 [Improvement](expr) fold child when const expr not folded (#38493) (#38961) 
cherry-pick from master 38493

1. fold child when const expr not folded
2. do not fold function `sleep`
3. move all exceptional expression into shouldSkipFold

before

mysql [test]>explain select sleep(sign(1)*100);
+-----------------------------------------------+
| Explain String(Nereids Planner)               |
+-----------------------------------------------+
| PLAN FRAGMENT 0                               |
|   OUTPUT EXPRS:                               |
|     sleep(cast((sign(1.0) * 100) as INT))[#0] |
|   PARTITION: UNPARTITIONED                    |
|                                               |
|   HAS_COLO_PLAN_NODE: false                   |
|                                               |
|   VRESULT SINK                                |
|      MYSQL_PROTOCAL                           |
|                                               |
|   0:VUNION(32)                                |
|      constant exprs:                          |
|          sleep(CAST((sign(1) * 100) AS int))  |
+-----------------------------------------------+
13 rows in set (15.02 sec)

mysql [test]>select sleep(sign(1)*100);
+-----------------------------------------------------+
| sleep(cast((sign(cast(1 as DOUBLE)) * 100) as INT)) |
+-----------------------------------------------------+
|                                                   1 |
+-----------------------------------------------------+
1 row in set (1 min 55.34 sec)


after

mysql [test]>explain select sleep(sign(1)*100);
+---------------------------------+
| Explain String(Nereids Planner) |
+---------------------------------+
| PLAN FRAGMENT 0                 |
|   OUTPUT EXPRS:                 |
|     sleep(100)[#0]              |
|   PARTITION: UNPARTITIONED      |
|                                 |
|   HAS_COLO_PLAN_NODE: false     |
|                                 |
|   VRESULT SINK                  |
|      MYSQL_PROTOCAL             |
|                                 |
|   0:VUNION(32)                  |
|      constant exprs:            |
|          sleep(100)             |
+---------------------------------+
13 rows in set (0.23 sec)

mysql [test]> select sleep(sign(1)*100);
+-----------------------------------------------------+
| sleep(cast((sign(cast(1 as DOUBLE)) * 100) as INT)) |
+-----------------------------------------------------+
|                                                   1 |
+-----------------------------------------------------+
1 row in set (1 min 40.37 sec)


Co-authored-by: Pxl <pxl290@qq.com>
 2024-08-12 15:13:48 +08:00
lihangyu
635ea39aaf [fix](regression-test) fix output of jsonb (#39217) 
introduced by  #39025
 2024-08-12 15:07:10 +08:00
Jerry Hu
96260d97bd [fix](ub) undefined behavior in FixedContainer (#39191) (#39201) 
## Proposed changes

pick #39191
Undefined behavior occurs if there is a null value in the list.

```
/root/doris/be/src/vec/common/string_ref.h:271:54: runtime error: null pointer passed as argument 2, which is declared to never be null
/var/local/ldb-toolchain/bin/../usr/include/string.h:64:33: note: nonnull attribute specified here
#0 0x5616d072245d in doris::StringRef::eq(doris::StringRef const&) const /root/doris/be/src/vec/common/string_ref.h:271:41
#1 0x5616d072245d in doris::StringRef::operator==(doris::StringRef const&) const /root/doris/be/src/vec/common/string_ref.h:274:60
#2 0x5616d072245d in doris::FixedContainer::find(doris::StringRef const&) const /root/doris/be/src/exprs/hybrid_set.h:76:36
#3 0x5616d072245d in void doris::StringValueSet>::_find_batch(doris::vectorized::IColumn const&, unsigned long, doris::vectorized::PODArray, 16ul, 15ul> const*, doris::vectorized::PODArray, 16ul, 15ul>&) /root/doris/be/src/exprs/hybrid_set.h:688:63
#4 0x5616d0747857 in doris::vectorized::FunctionIn::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long) const /root/doris/be/src/vec/functions/in.h:170:21
#5 0x5616c741fa3a in doris::vectorized::DefaultExecutable::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long) const /root/doris/be/src/vec/functions/function.h:462:26
#6 0x5616cbb5b650 in doris::vectorized::PreparedFunctionImpl::_execute_skipped_constant_deal(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp
#7 0x5616cbb4e14e in doris::vectorized::PreparedFunctionImpl::execute_without_low_cardinality_columns(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp:244:12
#8 0x5616cbb4e3c2 in doris::vectorized::PreparedFunctionImpl::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp:250:12
#9 0x5616c741cd68 in doris::vectorized::IFunctionBase::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.h:190:19
#10 0x5616c74cf712 in doris::vectorized::VInPredicate::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vin_predicate.cpp:130:5
#11 0x5616c740d5c0 in doris::vectorized::VectorizedFnCall::_do_execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*, std::vector>&) /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:183:9
#12 0x5616c740ecf5 in doris::vectorized::VectorizedFnCall::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:215:12
#13 0x5616c7462e24 in doris::vectorized::VCompoundPred::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vcompound_pred.h:127:38
#14 0x5616c74bccec in doris::vectorized::VExprContext::execute(doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vexpr_context.cpp:54:5
#15 0x5616c74c1dcc in doris::vectorized::VExprContext::execute_conjuncts(std::vector, std::allocator>> const&, std::vector, 16ul, 15ul>, std::allocator, 16ul, 15ul>>> const*, bool, doris::vectorized::Block*, doris::vectorized::PODArray, 16ul, 15ul>, bool) /root/doris/be/src/vec/exprs/vexpr_context.cpp:169:9
#16 0x5616c74c5108 in doris::vectorized::VExprContext::execute_conjuncts_and_filter_block(std::vector, std::allocator>> const&, doris::vectorized::Block*, std::vector>&, int, doris::vectorized::PODArray, 16ul, 15ul>&) /root/doris/be/src/vec/exprs/vexpr_context.cpp:322:5
#17 0x5616ad8a7f1a in doris::segment_v2::SegmentIterator::_execute_common_expr(unsigned short*, unsigned short&, doris::vectorized::Block*) /root/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2680:5
#18 0x5616ad89e86e in doris::segment_v2::SegmentIterator::_next_batch_internal(doris::vectorized::Block*) /root/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2582:25
#19 0x5616ad892f5c in doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*)::$_0::operator()() const /root/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2315:9
#20 0x5616ad892f5c in doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*) /root/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2314:19
#21 0x5616ad6dd9cc in doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::vectorized::Block*) /root/doris/be/src/olap/rowset/segment_v2/lazy_init_segment_iterator.h:44:33
#22 0x5616ad269d67 in doris::BetaRowsetReader::next_block(doris::vectorized::Block*) /root/doris/be/src/olap/rowset/beta_rowset_reader.cpp:380:29
#23 0x5616de6de110 in doris::vectorized::VCollectIterator::Level0Iterator::_refresh() /root/doris/be/src/vec/olap/vcollect_iterator.h
#24 0x5616de6c967f in doris::vectorized::VCollectIterator::Level0Iterator::refresh_current_row() /root/doris/be/src/vec/olap/vcollect_iterator.cpp:514:24
#25 0x5616de6ca8a6 in doris::vectorized::VCollectIterator::Level0Iterator::ensure_first_row_ref() /root/doris/be/src/vec/olap/vcollect_iterator.cpp:493:14
#26 0x5616de6d7008 in doris::vectorized::VCollectIterator::Level1Iterator::ensure_first_row_ref() /root/doris/be/src/vec/olap/vcollect_iterator.cpp:692:27
#27 0x5616de6bd200 in doris::vectorized::VCollectIterator::build_heap(std::vector, std::allocator>>&) /root/doris/be/src/vec/olap/vcollect_iterator.cpp:186:9
#28 0x5616de651b6c in doris::vectorized::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) /root/doris/be/src/vec/olap/block_reader.cpp:157:5
#29 0x5616de65526f in doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) /root/doris/be/src/vec/olap/block_reader.cpp:229:19
#30 0x5616e175a0f9 in doris::vectorized::NewOlapScanner::open(doris::RuntimeState*) /root/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:237:32
#31 0x5616c736ad34 in doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr, std::shared_ptr) /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:236:5
#32 0x5616c736f05e in doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()::operator()() const::'lambda'()::operator()() const /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:176:21
#33 0x5616c736f05e in doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()::operator()() const /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:175:31
#34 0x5616c736f05e in void std::_invoke_impl, std::shared_ptr)::$_1::operator()() const::'lambda'()&>(std::_invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
#35 0x5616c736f05e in std::enable_if, std::shared_ptr)::$1::operator()() const::'lambda'()&>, void>::type std::_invoke_r, std::shared_ptr)::$_1::operator()() const::'lambda'()&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2
#36 0x5616c736f05e in std::_Function_handler, std::shared_ptr)::$_1::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9
#37 0x5616aeed6a3b in doris::ThreadPool::dispatch_thread() /root/doris/be/src/util/threadpool.cpp:543:24
#38 0x5616aeeae4f7 in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:498:5
#39 0x7f7e663e3ac2 in start_thread nptl/pthread_create.c:442:8
#40 0x7f7e6647584f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /root/doris/be/src/vec/common/string_ref.h:271:54 in
```

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-12 10:02:07 +08:00
Mryange
ebf5d70c9d [fix](function) MicroSecondsSub without scale (#38945) (#39194) 
## Proposed changes
https://github.com/apache/doris/pull/38945
Added the computeSignature function for millisecond/microsecond
calculation functions to generate parameters and return values with the
appropriate precision.
Modified the microSecondsAdd function, which was used for constant
folding, because constant folding uses the precision of the parameters
for calculation. However, for millisecond/microsecond calculations, it
is necessary to set the precision to the maximum to ensure correct
display.


before
```
mysql> SELECT MICROSECONDS_SUB('2010-11-30 23:50:50', 2);
+-------------------------------------------------------------------+
| microseconds_sub(cast('2010-11-30 23:50:50' as DATETIMEV2(0)), 2) |
+-------------------------------------------------------------------+
| 2010-11-30 23:50:49                                               |
+-------------------------------------------------------------------+
```
now
```
mysql> SELECT MICROSECONDS_SUB('2010-11-30 23:50:50', 2);
+-------------------------------------------------------------------+
| microseconds_sub(cast('2010-11-30 23:50:50' as DATETIMEV2(0)), 2) |
+-------------------------------------------------------------------+
| 2010-11-30 23:50:49.999998                                        |
+-------------------------------------------------------------------+
```


<!--Describe your changes.-->

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-12 10:01:28 +08:00
amory
b38caed808 [Improve](columns)replace fatal with exception #38035 (#38996) 2024-08-12 09:51:30 +08:00
daidai
3da2d1c9d6 [bug](parquet)Fix the problem that the parquet reader reads the missing sub-columns of the struct and fails. (#38718) (#39192) 
bp #38718
 2024-08-11 20:37:40 +08:00
hui lai
0ee0dd6ae3 [fix](routine load) reset Kafka progress cache when routine load job topic change (#38474) (#39181) 
pick (#38474)

When change routine load job topic from test_topic_before to
test_topic_after by
```
ALTER ROUTINE LOAD FOR test_topic_change FROM KAFKA("kafka_topic" = "test_topic_after");
```
(test_topic_before has 5 rows and test_topic_after has 1 rows)

Exception happened, which cannot consume any data:
```
2024-07-29 15:57:28,122 WARN (Routine load task scheduler|55) [KafkaRoutineLoadJob.hasMoreDataToConsume():792] Kafka offset fallback. partition: 0, cache offset: 5 get latest of
fset: 1, task 16656914-ba0a-465d-8e79-8252b423b0fc, job 16615
2024-07-29 15:57:28,123 WARN (Routine load task scheduler|55) [KafkaRoutineLoadJob.hasMoreDataToConsume():792] Kafka offset fallback. partition: 0, cache offset: 5 get latest of
fset: 1, task 16656914-ba0a-465d-8e79-8252b423b0fc, job 16615
2024-07-29 15:57:28,125 WARN (Routine load task scheduler|55) [KafkaRoutineLoadJob.hasMoreDataToConsume():792] Kafka offset fallback. partition: 0, cache offset: 5 get latest of
fset: 1, task 16656914-ba0a-465d-8e79-8252b423b0fc, job 16615
2024-07-29 15:57:28,126 WARN (Routine load task scheduler|55) [KafkaRoutineLoadJob.hasMoreDataToConsume():792] Kafka offset fallback. partition: 0, cache offset: 5 get latest of
fset: 1, task 16656914-ba0a-465d-8e79-8252b423b0fc, job 16615
2024-07-29 15:57:28,128 WARN (Routine load task scheduler|55) [KafkaRoutineLoadJob.hasMoreDataToConsume():792] Kafka offset fallback. partition: 0, cache offset: 5 get latest of
fset: 1, task 16656914-ba0a-465d-8e79-8252b423b0fc, job 16615
2024-07-29 15:57:28,129 WARN (Routine load task scheduler|55) [KafkaRoutineLoadJob.hasMoreDataToConsume():792] Kafka offset fallback. partition: 0, cache offset: 5 get latest of
fset: 1, task 16656914-ba0a-465d-8e79-8252b423b0fc, job 16615
2024-07-29 15:57:28,131 WARN (Routine load task scheduler|55) [KafkaRoutineLoadJob.hasMoreDataToConsume():792] Kafka offset fallback. partition: 0, cache offset: 5 get latest of
fset: 1, task 16656914-ba0a-465d-8e79-8252b423b0fc, job 16615
2024-07-29 15:57:28,133 WARN (Routine load task scheduler|55) [KafkaRoutineLoadJob.hasMoreDataToConsume():792] Kafka offset fallback. partition: 0, cache offset: 5 get latest of
fset: 1, task 16656914-ba0a-465d-8e79-8252b423b0fc, job 16615
2024-07-29 15:57:28,134 WARN (Routine load task scheduler|55) [KafkaRoutineLoadJob.hasMoreDataToConsume():792] Kafka offset fallback. partition: 0, cache offset: 5 get latest of
fset: 1, task 16656914-ba0a-465d-8e79-8252b423b0fc, job 16615
2024-07-29 15:57:28,136 WARN (Routine load task scheduler|55) [KafkaRoutineLoadJob.hasMoreDataToConsume():792] Kafka offset fallback. partition: 0, cache offset: 5 get latest of
fset: 1, task 16656914-ba0a-465d-8e79-8252b423b0fc, job 16615
2024-07-29 15:57:28,137 WARN (Routine load task scheduler|55) [KafkaRoutineLoadJob.hasMoreDataToConsume():792] Kafka offset fallback. partition: 0, cache offset: 5 get latest of
fset: 1, task 16656914-ba0a-465d-8e79-8252b423b0fc, job 16615
```

It is necessary to reset Kafka progress cache when routine load job
topic change.
 2024-08-10 23:00:39 +08:00
yangshijie
5f77f909d9 [cherry-pick](branch-2.1) Pick "[feature](function) support ip functions named ipv4_to_ipv6 and cut_ipv6" (#39058) 
## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
pick https://github.com/apache/doris/pull/36883 and
https://github.com/apache/doris/pull/35239
 2024-08-10 18:37:11 +08:00
hui lai
0db158386a [chore](rowset writer) print rowset rows number when meet too many segments (#39091) (#39182) 
pick (#39091)
 2024-08-10 18:36:24 +08:00
zhangdong
9b4354fcb7 [fix](mtmv) second level MTMV always refresh all partition by mistake… (#39167) 
pick: 
https://github.com/apache/doris/pull/38698
 2024-08-10 18:26:56 +08:00
zhangdong
4dd9d4e1dc [enhance](mtmv) change mysql table type of mtmv to table (#38797) (#39166) 
pick: https://github.com/apache/doris/pull/38797
 2024-08-10 18:20:48 +08:00
seawinde
878ce29fa7 [fix](mtmv) Fix rewrite by materialized view fail when query hive table (#38909) (#39163) 
## Proposed changes

commitId: 08c9e051
pr: https://github.com/apache/doris/pull/38909
 2024-08-10 18:15:11 +08:00
zhangdong
df45398912 [fix](mtmv)fix can not show create mtmv use follower fe (#38794) (#39162) 
pick: https://github.com/apache/doris/pull/38794
 2024-08-10 18:12:01 +08:00
zhangdong
29ad364007 [enhance](mtmv)Disable mtmv list rollup (#38124) (#39158) 
pick master: https://github.com/apache/doris/pull/38124
 2024-08-10 18:06:38 +08:00
feiniaofeiafei
3dc150a0da [Fix](nereids) fix bind expression compare dbname ignore cluster (#39114) (#39142) 
cherry-pick #39114 to branch-2.1
 2024-08-10 18:00:48 +08:00
zzzxl
5e1e725cee [feature](inverted index) Add multi_match function #37722 #38931 #39149 (#38877) 2024-08-10 15:20:08 +08:00
yujun
80b6345270 [fix](create table) fix nereids create table no check buckets=0 #38971 (#38999) 
cherry pick from #38971
 2024-08-09 21:34:35 +08:00
Kaijie Chen
1a3f547d35 [fix](move-memtable) ignore single replica load when move memtable (#32845) (#39154) 
backport #32845

Co-authored-by: Xin Liao <liaoxinbit@126.com>
 2024-08-09 20:25:05 +08:00
924060929
3f6c71e47b [enhancement](Nereids) fast compute hash code of deep expression tree to reduce conflict (#38981) (#39133) 
The Expression.hashCode default is getClass().hashCode(), just contains one level information, so the lots of expressions which is same type will return the same hash code and conflict, then compare deeply in the HashMap cause inefficient and hold table lock for long time.

This pr support fast compute hash code by the bottom literal and slot, reduce the compare expression time because of the conflict of hash code

In my test case, the sql planner time can reduce from 20 minutes(not finished) to 35 seconds
 2024-08-09 16:28:24 +08:00
zhannngchen
8a682d43ec [fix](ut) repair segcompaction ut (#38165) (#38225) 
cherry-pick #38165
 2024-08-09 15:52:18 +08:00
Mingyu Chen
4aae68f62f [test](catalog) add upgrade test cases for external catalog (#39063) (#39140) 
bp #39063
 2024-08-09 15:12:26 +08:00
walter
b6faf3910e [fix](gson) Fix GSON pretty printting setting #37211 (#39135) 
cherry pick from #37211
 2024-08-09 15:06:48 +08:00