6dddd4c499
[function](cast)Make string casting to integers more like MySQL's beh… ( #41541 )
...
…avior (#38847 )
https://github.com/apache/doris/pull/38847
## Proposed changes
There are two issues here. First, the results of casting are
inconsistent between FE and BE .
```
FE
mysql [(none)]>select cast('3.000' as int);
+----------------------+
| cast('3.000' as INT) |
+----------------------+
| 3 |
+----------------------+
mysql [(none)]>set debug_skip_fold_constant = true;
BE
mysql [(none)]>select cast('3.000' as int);
+----------------------+
| cast('3.000' as INT) |
+----------------------+
| NULL |
+----------------------+
```
The second issue is that casting on BE converts '3.0' to null. Here, the
casting logic for FE and BE has been unified
<!--Describe your changes.-->
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
---------
Co-authored-by: Xinyi Zou <zouxinyi02@gmail.com >
2024-10-11 09:32:00 +08:00
4c9ebbb3b9
[fix](cloud) cloud group commit should skip repaly wal if label is already used and the txn state is committed or visible ( #41262 ) ( #41461 )
...
pick https://github.com/apache/doris/pull/41262
2024-10-10 22:27:04 +08:00
f2ba1f2fb3
[bugfix](segmentload) should remove segment from segment cache if load segment failed ( #41608 ) ( #41660 )
2024-10-10 19:40:22 +08:00
0fb42d3a48
[Enhancement](tvf)catalog tvf implements user permission checks and hides sensitive information ( #41497 ) ( #41604 )
...
bp #41497
before #21790
## Proposed changes
This PR unifies the duplicate parts of `catalog tvf` and `show
catalogs`, adds permission check when querying `catalog tvf`, and hides
sensitive information.
2024-10-10 17:55:40 +08:00
1db0aef9b7
[feature](array_agg) support array_agg with param is array/map/struct… ( #41651 )
...
… (#40697 )
this pr we support array_agg function support param with array map
struct type
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
2024-10-10 17:54:54 +08:00
3120bfb6e3
[fix](pipelinex) fix fragment instance progress reports (part 2) ( #40694 ) ( #41641 )
...
backport #40694
2024-10-10 17:49:41 +08:00
30492a2438
[opt](load) print more detailed log when stream load finished #41398 ( #41639 )
...
cherry pick from #41398
2024-10-10 17:47:48 +08:00
d32688e091
[Enhancement](multi-catalog) Set hdfs native client logger to glog and redirect jvm stdout/stderr logger to jni.log. ( #41633 )
...
Backport #39540 .
Co-authored-by: Mingyu Chen <morningman@163.com >
2024-10-10 17:47:21 +08:00
a26079c09d
[Opt](load) Optimize the error messages of -235 and -238 for loading #41048 ( #41638 )
...
cherry pick from #41048
2024-10-10 14:20:52 +08:00
33fad04341
[opt](Nereids) use 1 instead narrowest column when do column pruning ( #41548 ) ( #41627 )
...
pick from master #41548
2024-10-10 14:02:23 +08:00
aa541fddf9
[fix](load) disable num segments check in compatibility mode ( #41053 ) ( #41552 )
...
backport #41053
2024-10-10 11:20:16 +08:00
e218fd2314
[Fix](inverted index) add DATEV2 and DATETIMEV2 for inverted index reader #41565 ( #41579 )
...
cherry pick from #41565
2024-10-09 15:32:41 +08:00
31b506c8cc
[Enhancement](inverted index) return OK instead of not supported in expr evaluate_inverted_index #41567 ( #41578 )
...
cherry pick from #41567
2024-10-09 15:14:38 +08:00
0185f8069f
[fix](crash) fix be crash because of int overflow ( #41554 ) ( #41568 )
2024-10-09 14:20:55 +08:00
9fe77b335c
[Enhancement](inverted index) apply inverted index when has any #41547 ( #41584 )
...
cherry pick from #41547
2024-10-09 14:13:38 +08:00
afb477c66d
[Fix](inverted index) Fix wrong need read data opt when enable_common_expr_pushdown is disabled #40689 ( #41562 )
...
cherry pick from #40689
2024-10-08 22:12:10 +08:00
c24ff2ff81
[fix](upgrade) fix version check failure of window_funnel when upgrading ( #41542 )
...
## Proposed changes
Issue Number: close #xxx
Fix fix version check failure of window_funnel when upgrading from 2.1.6
and higher version to latest branch 2.1.
```
02:49:13 F20240930 02:47:48.546983 7581 block.cpp:89] Check failed: BeExecVersionManager::check_be_exec_version(be_exec_version)
02:49:13 *** Check failure stack trace: ***
02:49:13 @ 0x564640041856 google::LogMessage::SendToLog()
02:49:13 @ 0x56464003e2a0 google::LogMessage::Flush()
02:49:13 @ 0x564640042099 google::LogMessageFatal::~LogMessageFatal()
02:49:13 @ 0x56463922d106 doris::vectorized::Block::deserialize()
02:49:13 @ 0x5646390a82bf doris::vectorized::WindowFunnelState<>::read()
02:49:13 @ 0x5646390a6889 doris::vectorized::IAggregateFunctionDataHelper<>::deserialize_and_merge()
02:49:13 @ 0x5646390acdc3 doris::vectorized::IAggregateFunctionHelper<>::deserialize_and_merge_from_column_range()
02:49:13 @ 0x56463fa77152 doris::pipeline::AggSinkLocalState::_merge_without_key()
02:49:13 @ 0x56463fa9d114 doris::pipeline::AggSinkLocalState::Executor<>::execute()
02:49:13 @ 0x56463fa78569 doris::pipeline::AggSinkOperatorX::sink()
02:49:13 @ 0x564640013296 doris::pipeline::PipelineXTask::execute()
02:49:13 @ 0x56464001d41c doris::pipeline::TaskScheduler::_do_work()
02:49:13 @ 0x56463663e078 doris::ThreadPool::dispatch_thread()
02:49:13 @ 0x564636634901 doris::Thread::supervise_thread()
02:49:13 @ 0x7fb64cf58ac3 (unknown)
02:49:13 @ 0x7fb64cfea850 (unknown)
02:49:13 @ (nil) (unknown)
02:49:13 *** Query id: b0cd194940184766-961c310e833e92b1 ***
02:49:13 *** is nereids: 1 ***
02:49:13 *** tablet id: 0 ***
02:49:13 *** Aborted at 1727635668 (unix time) try "date -d @1727635668" if you are using GNU date ***
02:49:13 *** Current BE git commitID: 653e315ba5 ***
02:49:13 *** SIGABRT unknown detail explain (@0x1648) received by PID 5704 (TID 7581 OR 0x7fb354a9a640) from PID 5704; stack trace: ***
02:49:13 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/common/signal_handler.h:421
02:49:13 1# 0x00007FB64CF06520 in /lib/x86_64-linux-gnu/libc.so.6
02:49:13 2# pthread_kill at ./nptl/pthread_kill.c:89
02:49:13 3# raise at ../sysdeps/posix/raise.c:27
02:49:13 4# abort at ./stdlib/abort.c:81
02:49:13 5# 0x000056464004C06D in /mnt/hdd01/ci/compatibility-deploy/be/lib/doris_be
02:49:13 6# 0x000056464003E76A in /mnt/hdd01/ci/compatibility-deploy/be/lib/doris_be
02:49:13 7# google::LogMessage::SendToLog() in /mnt/hdd01/ci/compatibility-deploy/be/lib/doris_be
02:49:13 8# google::LogMessage::Flush() in /mnt/hdd01/ci/compatibility-deploy/be/lib/doris_be
02:49:13 9# google::LogMessageFatal::~LogMessageFatal() in /mnt/hdd01/ci/compatibility-deploy/be/lib/doris_be
02:49:13 10# doris::vectorized::Block::deserialize(doris::PBlock const&) at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/vec/core/block.cpp:113
02:49:13 11# doris::vectorized::WindowFunnelState<(doris::vectorized::TypeIndex)14, long>::read(doris::vectorized::BufferReadable&) at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/vec/aggregate_functions/aggregate_function_window_funnel.h:363
02:49:13 12# doris::vectorized::IAggregateFunctionDataHelper<doris::vectorized::WindowFunnelState<(doris::vectorized::TypeIndex)14, long>, doris::vectorized::AggregateFunctionWindowFunnel<(doris::vectorized::TypeIndex)14, long> >::deserialize_and_merge(char*, char*, doris::vectorized::BufferReadable&, doris::vectorized::Arena*) const at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/vec/aggregate_functions/aggregate_function.h:517
02:49:13 13# doris::vectorized::IAggregateFunctionHelper<doris::vectorized::AggregateFunctionNullVariadicInline<doris::vectorized::AggregateFunctionWindowFunnel<(doris::vectorized::TypeIndex)14, long>, false> >::deserialize_and_merge_from_column_range(char*, doris::vectorized::IColumn const&, unsigned long, unsigned long, doris::vectorized::Arena*) const at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/vec/aggregate_functions/aggregate_function.h:465
02:49:13 14# doris::pipeline::AggSinkLocalState::_merge_without_key(doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/pipeline/exec/aggregation_sink_operator.cpp:389
02:49:13 15# doris::pipeline::AggSinkLocalState::Executor<true, true>::execute(doris::pipeline::AggSinkLocalState*, doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/pipeline/exec/aggregation_sink_operator.h:73
02:49:13 16# doris::pipeline::AggSinkOperatorX::sink(doris::RuntimeState*, doris::vectorized::Block*, bool) at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/pipeline/exec/aggregation_sink_operator.cpp:744
02:49:13 17# doris::pipeline::PipelineXTask::execute(bool*) at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/pipeline/pipeline_x/pipeline_x_task.cpp:332
02:49:13 18# doris::pipeline::TaskScheduler::_do_work(unsigned long) at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/pipeline/task_scheduler.cpp:347
02:49:13 19# doris::ThreadPool::dispatch_thread() in /mnt/hdd01/ci/compatibility-deploy/be/lib/doris_be
02:49:13 20# doris::Thread::supervise_thread(void*) at /home/zcp/repo_center/doris_branch-2.1/doris/be/src/util/thread.cpp:499
02:49:13 21# start_thread at ./nptl/pthread_create.c:442
02:49:13 22# 0x00007FB64CFEA850 at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:83
02:49:13 [2024-09-30 02:49:13,147 __main__:796] [INFO]: 172.20.50.73 last coredump sql: 2024-09-30 02:48:18,328 [query] Query b0cd194940184766-961c310e833e92b1 1 times with new query id: 2e0e00de0e7548dd-95f9abc9d8d11c3a
```
2024-10-08 17:13:33 +08:00
cb24ccc112
[bugfix](brpc) Should use status to generate protobuf message, because it will encoding Backend Info ( #41515 ) ( #41522 )
...
Should use status to generate protobuf message, because it will encoding
Backend Info
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
2024-10-04 23:03:55 +08:00
d33a3eb1c5
[cherry-pick](branch-2.1) Pick "[Fix](LZ4 compression) Fix wrong LZ4 compression max input size limit ( #41239 )" ( #41505 )
...
## Proposed changes
LZ4 compression max supported value is LZ4_MAX_INPUT_SIZE, which is
0x7E000000(2,113,929,216 bytes). Doris use wrong max size INT_MAX, which
is 2,147,483,647, to check. If input data size is between this two size,
then it can pass the check but LZ4 compression will fail.
This PR fix it.
<!--Describe your changes.-->
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
2024-10-01 22:43:11 +08:00
98a1311aa2
[Opt](scanner-scheduler) Opt scanner scheduler starvation issue. ( #41484 )
...
## Proposed changes
Backport #40641
2024-09-30 15:40:20 +08:00
2b9c963edb
[fix](scanner) Check query status when iterating through rowsets and segments #41363 ( #41452 )
...
cherry pick from #41363
2024-09-30 09:49:46 +08:00
676c4e3b43
[Fix](inverted index) fix comparison for string after refactor compound #40338 ( #41441 )
...
cherry pick from #40338
2024-09-29 22:10:01 +08:00
34429bfa0e
[Chore](inverted index) remove useless code of compound filters for inverted index #40258 ( #41448 )
...
cherry pick from #40258
2024-09-29 17:27:29 +08:00
d1ee0f4e74
[Fix](inverted index) fix profile metrics for inverted index after refactor #40324 ( #41440 )
...
cherry pick from #40324
2024-09-29 14:10:54 +08:00
5ab174b7dc
[Chore](inverted index) remove useless code in scan operator for normalize compound and match predicate #40232 ( #41449 )
...
cherry pick from #40232
2024-09-29 12:30:32 +08:00
ddbb436e36
[Chore][inverted index] remove duplicate null bitmap reader in function array index #40907 ( #41444 )
...
cherry pick from #40907
2024-09-29 12:09:12 +08:00
b3bb4f4402
[Enhancement](inverted index) improve expr evaluate_inverted_index performace and remove useless code #40600 ( #41442 )
...
cherry pick from #40600
2024-09-29 12:00:58 +08:00
d659750fd9
[pick](Serde-2.1) fix variant serde may lost num_rows when subcolumns empty ( #41438 )
...
serialization object with empty subcolumns may lost num_rows, so need to
record num_rows and set back num_rows in serdes
backport #38413
2024-09-29 09:45:37 +08:00
727f0374be
[Refactor](inverted index) refactor inverted index compound predicates evaluate logic #38908 ( #41385 )
...
cherry pick from #38908
2024-09-29 09:19:17 +08:00
0b4552f74b
[cherry-pick](branch-2.1) pick hive text write from master ( #40537 )
...
## Proposed changes
pick prs:
https://github.com/apache/doris/pull/38549
https://github.com/apache/doris/pull/40183
https://github.com/apache/doris/pull/40315
---------
Co-authored-by: Calvin Kirs <kirs@apache.org >
2024-09-27 20:57:07 +08:00
e0c9cbde49
[opt](scanner) Add scanner metrics #40496 ( #41314 )
...
cherry pick from #40496
2024-09-27 09:56:51 +08:00
eb13cd4154
[branch-2.1] Picks "[Fix](partial update) Fix __DORIS_SEQUENCE_COL__ is not set for newly inserted rows in partial update #40272 " ( #40964 )
...
picks https://github.com/apache/doris/pull/40272
2024-09-26 22:54:27 +08:00
bf3d4240be
[fix](window_func) fix bug of agg function used in window function and add many test cases ( #40678 ) ( #41328 )
...
## Proposed changes
Issue Number: close #xxx
BP #40678
2024-09-26 22:50:34 +08:00
9ec54e9eb5
[Fix](Serde-2.1) fix potential mem leak in array serde write_one_cell_to_json ( #41339 )
...
placement new may lead to mem leak in Field without calling it's
desctructor
2024-09-26 22:49:34 +08:00
d875b026b9
[fix](snapshot) Link binlog files according to consistent rowsets #41319 ( #41345 )
...
cherry pick from #41319
2024-09-26 21:47:21 +08:00
a11fd62043
[fix](window function) Fix illegal frame range ( #41147 ) ( #41305 )
...
pick #41147
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int,
siginfo_t*, void*) at
/home/zcp/repo_center/doris_master/doris/be/src/common/signal_handler.h:421
1# 0x00007F591D573520 in /lib/x86_64-linux-gnu/libc.so.6
2# pthread_kill at ./nptl/pthread_kill.c:89
3# raise at ../sysdeps/posix/raise.c:27
4# abort at ./stdlib/abort.c:81
5# _nl_load_domain at ./intl/loadmsgcat.c:1177
6# 0x00007F591D56AE96 in /lib/x86_64-linux-gnu/libc.so.6
7# doris::vectorized::PODArray<unsigned char, 4096ul, Allocator<false,
false, false, DefaultMemoryAllocator>, 16ul, 15ul>::operator[](long)
const at
/home/zcp/repo_center/doris_master/doris/be/src/vec/common/pod_array.h:365
8# doris::vectorized::ColumnNullable::is_null_at(unsigned long) const at
/home/zcp/repo_center/doris_master/doris/be/src/vec/columns/column_nullable.h:158
9#
doris::vectorized::ReaderFirstAndLastData<doris::vectorized::ColumnVector<double>,
true, true, false>::insert_result_into(doris::vectorized::IColumn&)
const at
/home/zcp/repo_center/doris_master/doris/be/src/vec/aggregate_functions/aggregate_function_reader_first_last.h:125
10# doris::pipeline::AnalyticLocalState::_insert_result_info(long) in
/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be
11# std::_Function_handler<void (long), std::_Bind_result<void, void
(doris::pipeline::AnalyticLocalState::*(doris::pipeline::AnalyticLocalState*,
std::_Placeholder<1>))(long)> >::_M_invoke(std::_Any_data const&,
long&&) at
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
12# std::function<void (long)>::operator()(long) const at
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560
13# doris::pipeline::AnalyticLocalState::_get_next_for_rows(unsigned
long) in /mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be 14#
std::enable_if<is_invocable_r_v<doris::Status, doris::Status
(doris::pipeline::AnalyticLocalState::*&)(unsigned long),
doris::pipeline::AnalyticLocalState*&, unsigned long>,
doris::Status>::type std::__invoke_r<doris::Status, doris::Status
(doris::pipeline::AnalyticLocalState::*&)(unsigned long),
doris::pipeline::AnalyticLocalState*&, unsigned long>(doris::Status
(doris::pipeline::AnalyticLocalState::*&)(unsigned long),
doris::pipeline::AnalyticLocalState*&, unsigned long&&) at
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:114
15# doris::Status std::_Bind_result<doris::Status, doris::Status
(doris::pipeline::AnalyticLocalState::*(doris::pipeline::AnalyticLocalState*,
std::_Placeholder<1>))(unsigned long)>::__call<doris::Status, unsigned
long&&, 0ul, 1ul>(std::tuple<unsigned long&&>&&, std::_Index_tuple<0ul,
1ul>) at
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:570
16# std::_Function_handler<doris::Status (unsigned long),
std::_Bind_result<doris::Status, doris::Status
(doris::pipeline::AnalyticLocalState::*(doris::pipeline::AnalyticLocalState*,
std::_Placeholder<1>))(unsigned long)> >::_M_invoke(std::_Any_data
const&, unsigned long&&) at
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
17# std::function<doris::Status (unsigned long)>::operator()(unsigned
long) const at
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560
18#
doris::pipeline::AnalyticSourceOperatorX::get_block(doris::RuntimeState*,
doris::vectorized::Block*, bool*) in
/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be
19#
doris::pipeline::OperatorXBase::get_block_after_projects(doris::RuntimeState*,
doris::vectorized::Block*, bool*) at
/home/zcp/repo_center/doris_master/doris/be/src/pipeline/exec/operator.cpp:322
20# doris::pipeline::PipelineTask::execute(bool*) in
/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be
21# doris::pipeline::TaskScheduler::_do_work(unsigned long) at
/home/zcp/repo_center/doris_master/doris/be/src/pipeline/task_scheduler.cpp:138
22# doris::ThreadPool::dispatch_thread() in
/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be
23# doris::Thread::supervise_thread(void*) at
/home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:499 24#
start_thread at ./nptl/pthread_create.c:442
25# 0x00007F591D657850 at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:83
2024-09-26 09:55:33 +08:00
11aab648a3
[fix](crash) be crash because of duplicate __PARTIAL_UPDATE_AUTO_INC_COLUMN__ ( #41172 ) ( #41187 )
2024-09-25 19:49:57 +08:00
869680bf38
[Improvement](sort) enlarge HEAP_SORT_THRESHOLD to 50000 ( #41272 )
...
## Proposed changes
enlarge HEAP_SORT_THRESHOLD to 50000
2024-09-25 17:30:00 +08:00
5b3b2cec80
[feat](metatable) support table$partitions for hive table ( #40774 ) ( #41230 )
...
bp #40774
and pick part of #34552 , add `isPartitionedTable()` interface in `TableIf`
2024-09-25 09:52:07 +08:00
8bb57bcc3e
[fix] (inverted index) fix the error in the query result when using count on index ( #41200 )
...
## Proposed changes
Introduced by #39473
<!--Describe your changes.-->
2024-09-24 19:47:18 +08:00
78adf64353
[compability](variant) fix nullable serde during upgrading to 3.0 ( #41191 )
...
need to check nullable during serialization when upgrading to 3.0
2024-09-24 19:44:54 +08:00
d4c1b39d03
[fix](multi table) restrict the multi tables load memory under high concurrency with a large number of tables ( #39992 ) ( #41131 )
...
pick (#39992 )
BE node was killed by OOM-killer when use multi table load under high
concurrency with a large number of tables(128 concurrency and every
concurrency load 200 tables).
This pr restricts the multi tables load memory under this issue. If
memory reaches hard limit, new task will be rejected and return
directly.
2024-09-24 16:34:32 +08:00
4237341e0d
[fix](move-memtable) do not execute close if create rowset failed when loading MOW table ( #40105 ) ( #41132 )
...
pick (#40105 )
Core dump happened when load to MOW table:
```
Check failure stack trace: ***
@ 0x55fae437d246 google::LogMessage::SendToLog()
@ 0x55fae4379c90 google::LogMessage::Flush()
@ 0x55fae437da89 google::LogMessageFatal::~LogMessageFatal()
@ 0x55faacf26bbf doris::BaseTablet::check_delete_bitmap_correctness()
@ 0x55fab05049ef doris::RowsetBuilder::commit_txn()
@ 0x55fab09026e8 doris::LoadStreamWriter::close()
@ 0x55fab089eff7 std::_Function_handler<>::_M_invoke()
@ 0x55fab0d14d7c doris::WorkThreadPool<>::work_thread()
@ 0x55fae76ae6f0 execute_native_thread_routine
@ 0x7fa32ea45ac3 (unknown)
@ 0x7fa32ead7850 (unknown)
@ (nil) (unknown)
Query id: a21981d5c8ef4113-84df9a5a8680e004 ***
is nereids: 0 ***
tablet id: 0 ***
Aborted at 1724668499 (unix time) try "date -d @1724668499" if you are using GNU date ***
Current BE git commitID: 2f848737c1 ***
SIGABRT unknown detail explain (@0x20db) received by PID 8411 (TID 9837 OR 0x7f9e42cfe640) from PID 8411; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_master/doris/be/src/common/signal_handler.h:421
1# 0x00007FA32E9F3520 in /lib/x86_64-linux-gnu/libc.so.6
2# pthread_kill at ./nptl/pthread_kill.c:89
3# raise at ../sysdeps/posix/raise.c:27
4# abort at ./stdlib/abort.c:81
5# 0x000055FAE4387B1D in /mnt/hdd01/ci/master-deploy/be/lib/doris_be
6# 0x000055FAE437A15A in /mnt/hdd01/ci/master-deploy/be/lib/doris_be
7# google::LogMessage::SendToLog() in /mnt/hdd01/ci/master-deploy/be/lib/doris_be
8# google::LogMessage::Flush() in /mnt/hdd01/ci/master-deploy/be/lib/doris_be
9# google::LogMessageFatal::~LogMessageFatal() in /mnt/hdd01/ci/master-deploy/be/lib/doris_be
10# doris::BaseTablet::check_delete_bitmap_correctness(std::shared_ptr, long, long, std::unordered_set, std::equal_to, std::allocator > const&, std::vector, std::allocator > >*) at /home/zcp/repo_center/doris_master/doris/be/src/olap/base_tablet.cpp:1152
11# doris::RowsetBuilder::commit_txn() at /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset_builder.cpp:316
12# doris::LoadStreamWriter::close() at /home/zcp/repo_center/doris_master/doris/be/src/runtime/load_stream_writer.cpp:311
13# std::_Function_handler::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
14# doris::WorkThreadPool::work_thread(int) at /home/zcp/repo_center/doris_master/doris/be/src/util/work_thread_pool.hpp:159
15# execute_native_thread_routine at ../../../../../libstdc+-v3/src/c+11/thread.cc:84
16# start_thread at ./nptl/pthread_create.c:442
17# 0x00007FA32EAD7850 at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:83
```
If create rowset failed,` calc_delete_bitmap_task` still could be
executed:
```
add segment failed load_id=5649413b98976f0d-a105b42749f561b0, txn_id=2, tablet_id=10088, status=[INTERNAL_ERROR]create row
set failed
...
submit calc delete bitmap task to executor, tablet_id: 10088, txn_id: 2
```
This PR skips close to avoid `submit_calc_delete_bitmap_task` if create
rowset failed when loading MOW table to solve this problem.
2024-09-24 16:02:22 +08:00
d7659ff34d
[fix](bytebuffer) fix allocate size improper in append_and_flush ( #40613 ) ( #41133 )
...
pick (#40613 )
fix allocate size improper in append_and_flush introduced by
https://github.com/apache/doris/pull/38960
2024-09-24 16:01:52 +08:00
602275d1fe
[improvement](stream load) display the time of receiving network data ( #40735 ) ( #41139 )
...
pick (#40735 )
Display the time of receiving network data.
For example:
```
Stream load result: {
"TxnId": 19,
"Label": "c10ee032-c19c-484f-80b2-16f6c1d0c209",
"Comment": "",
"TwoPhaseCommit": "false",
"Status": "Success",
"Message": "OK",
"NumberTotalRows": 6,
"NumberLoadedRows": 6,
"NumberFilteredRows": 0,
"NumberUnselectedRows": 0,
"LoadBytes": 60,
"LoadTimeMs": 32,
"BeginTxnTimeMs": 0,
"StreamLoadPutTimeMs": 4,
"ReadDataTimeMs": 0,
"WriteDataTimeMs": 4,
"ReceiveDataTimeMs": 6,
"CommitAndPublishTimeMs": 22
}
```
2024-09-24 16:00:28 +08:00
ddd17be874
[fix](migrate) Fix iterator returning too early #41168 ( #41189 )
...
cherry pick from #41168
2024-09-24 14:10:10 +08:00
2b427c316a
[feature](functions) impl scalar functions normal_cdf,to_iso8601,from_iso8601_date ( #40695 ) ( #41049 )
...
bp #40695
2024-09-24 09:52:39 +08:00
0d38a9a36d
[feature](restore) support atomic restore ( #41107 )
...
Cherry-pick #40353 , #40734 , #40817 , #40876 , #40921 , #41017 , #41083
2024-09-24 09:41:41 +08:00
f6917acd6a
[cherry-pick](branch2.1) Impl translate and url encode 2.1 ( #41051 )
...
## Proposed changes
pick https://github.com/apache/doris/pull/40567
some code about const folding should wait the pr picked:
https://github.com/apache/doris/pull/40441
2024-09-23 14:26:27 +08:00
011baeb2d2
[bugfix](allocatebytes) ignore null ptr column in Block ( #41093 ) ( #41099 )
...
## Proposed changes
Sometimes if expr failed, then there will be a nullptr column left in
the block.
We should ignore the nullptr column, or exception will be thrown and
some profile will not computed correctly.
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
Co-authored-by: yiguolei <yiguolei@gmail.com >
2024-09-23 12:18:22 +08:00
