ae88d032db
[chore](ddl) support force_enable_feature_binlog #41796 ( #42926 )
...
cherry pick from #41796
2024-10-31 09:53:45 +08:00
fce4695f37
[Configuration](transactional-hive) Add skip_checking_acid_version_file session var to skip checking acid version file in some hive envs. ( #42111 )( #42225 ) ( #42939 )
...
cherry-pick (#42111 )(#42225 )
---------
Co-authored-by: Qi Chen <kaka11.chen@gmail.com >
2024-10-31 09:52:20 +08:00
0d008b5a43
[feat](nereids)disable join reorder if column stats is invalid #41790 (branch-2.1) ( #42902 )
...
## Proposed changes
pick #41790
Issue Number: close #xxx
<!--Describe your changes.-->
2024-10-30 23:47:03 +08:00
6d23020cb8
[fix](regression) do not test "set global sessionVar" on multi-fe cluster ( #42922 )
...
## Proposed changes
fix unstable rt case: test_statistic_global_variable
Issue Number: close #xxx
<!--Describe your changes.-->
2024-10-30 21:35:32 +08:00
e43d255815
[fix](array) fix add_array_null_value for inverted index_builder ( #42899 )
...
## Proposed changes
Pick #42638 to doris branch 2.1
Issue Number: close #xxx
<!--Describe your changes.-->
2024-10-30 19:16:59 +08:00
53cccfb08c
[test](move memtable) fix move memtable injection case fail ( #42639 ) ( #42891 )
...
Fix move memtable injection case fail.
pick (#42639 )
2024-10-30 16:29:40 +08:00
2a6d6c15c5
[enhance](hive)hive event code optimization #42637 ( #42876 )
...
cherry pick from #42637
Co-authored-by: zhangdong <493738387@qq.com >
2024-10-30 12:52:21 +08:00
17d84dc88f
[enhance](paimon)paimon scanner code optimization #42606 ( #42875 )
...
cherry pick from #42606
Co-authored-by: zhangdong <493738387@qq.com >
2024-10-30 12:51:59 +08:00
5c6e5fa486
[cherry-pick](branch-2.1)fix the mutil threads tests ( #42669 )
...
pick from master:
https://github.com/apache/doris/pull/39617
2024-10-30 10:30:08 +08:00
848234bd92
[fix](exchange) query blocked by exchange reach limit ( #42853 )
...
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
2024-10-30 10:13:26 +08:00
057d733328
[fix](cases) fix alias conflict case problem of driver url error ( #42676 ) ( #42859 )
...
pick: #42676
change driver url to doris internal one to get driver process more
robust
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
2024-10-30 09:57:26 +08:00
ba4d01d93c
[branch-2.1](hint) add detail for replica missing msg ( #42546 )
2024-10-29 22:47:41 +08:00
2042726ceb
[Fix](regression case) fix some unstable regression cases ( #42601 )
...
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
2024-10-29 18:12:40 +08:00
233fad4815
[opt](variable) force update some variable by variable version ( #41607 ) ( #42643 )
...
pick from master #41607
variable version:
000-100: doris-2.0.x
100-200: doris-2.1.x
200-300: doris-3.0.x
update variables
000:
nereids_timeout_second = 30 if original value is 5
100:
enable_nereids_dml = true
enable_nereids_dml_with_pipeline = true
enable_nereids_planner = true
enable_fallback_to_original_planner = true
enable_pipeline_x_engine = true
2024-10-29 15:57:14 +08:00
1538b82221
[Impl](Nereids) add nereids gc cost time and be fold const cost time to profile ( #42007 ) ( #42516 )
...
pick:#42007
add Nereids GarbageCollect Time and Nereids BeFoldConst Time to profile
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
2024-10-29 15:16:40 +08:00
42daf76a05
[fix](coalesce) fix 'heap-use-after-free' of function coalesce ( #42666 ) ( #42667 )
...
## Proposed changes
BP #42666
Should not modify state of vexpr when executing the expr, since it may
be executed in parallel.
Issue introduced by https://github.com/apache/doris/pull/42136
```
==75397==ERROR: AddressSanitizer: heap-use-after-free on address 0x60700294b060 at pc 0x56507fb8a370 bp 0x7fc861cd7050 sp 0x7fc861cd7048
19:58:49 READ of size 8 at 0x60700294b060 thread T1463 (Pipe_normal [wo)
19:58:49 #0 0x56507fb8a36f in doris::vectorized::FunctionCoalesce::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long) const /root/doris/be/src/vec/functions/function_coalesce.cpp:163:13
19:58:49 #1 0x56507958edfe in doris::vectorized::DefaultExecutable::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long) const /root/doris/be/src/vec/functions/function.h:463:26
19:58:49 #2 0x56507dbcbf5c in doris::vectorized::PreparedFunctionImpl::_execute_skipped_constant_deal(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp
19:58:49 #3 0x56507dbc65e3 in doris::vectorized::PreparedFunctionImpl::execute_without_low_cardinality_columns(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp:245:12
19:58:49 #4 0x56507dbc683b in doris::vectorized::PreparedFunctionImpl::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp:251:12
19:58:49 #5 0x56507958c81c in doris::vectorized::IFunctionBase::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.h:194:19
19:58:49 #6 0x565079578bee in doris::vectorized::VectorizedFnCall::_do_execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*, std::vector<unsigned long, std::allocator<unsigned long> >&) /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:188:5
19:58:49 #7 0x56507957a3e4 in doris::vectorized::VectorizedFnCall::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:203:12
19:58:49 #8 0x56507964994a in doris::vectorized::VExprContext::execute(doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vexpr_context.cpp:55:5
19:58:49 #9 0x56508f8d7496 in doris::pipeline::OperatorXBase::do_projections(doris::RuntimeState*, doris::vectorized::Block*, doris::vectorized::Block*) const /root/doris/be/src/pipeline/exec/operator.cpp:294:13
19:58:49 #10 0x56508f8d96ba in doris::pipeline::OperatorXBase::get_block_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*) /root/doris/be/src/pipeline/exec/operator.cpp:335:18
19:58:49 #11 0x565093c2560b in doris::pipeline::PipelineTask::execute(bool*) /root/doris/be/src/pipeline/pipeline_task.cpp:374:13
19:58:49 #12 0x565093c6a164 in doris::pipeline::TaskScheduler::_do_work(int) /root/doris/be/src/pipeline/task_scheduler.cpp:138:9
19:58:49 #13 0x565062529d1d in doris::ThreadPool::dispatch_thread() /root/doris/be/src/util/threadpool.cpp:543:24
19:58:49 #14 0x5650625021ce in std::function<void ()>::operator()() const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9
19:58:49 #15 0x5650625021ce in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:498:5
19:58:49 #16 0x7fca1844a608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8
19:58:49 #17 0x7fca186f7132 in __clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95
19:58:49
19:58:49 0x60700294b060 is located 16 bytes inside of 80-byte region [0x60700294b050,0x60700294b0a0)
19:58:49 freed by thread T1462 (Pipe_normal [wo) here:
19:58:49 #0 0x56505dd2f80d in operator delete(void*) (/mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be+0x2f79e80d) (BuildId: 60cc2a41212f4e75)
19:58:49 #1 0x565079596a6f in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:702:11
19:58:49 #2 0x565079596a6f in std::__shared_ptr<doris::vectorized::IFunctionBase, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1149:31
19:58:49 #3 0x565079596a6f in std::__shared_ptr<doris::vectorized::IFunctionBase, (__gnu_cxx::_Lock_policy)2>::operator=(std::__shared_ptr<doris::vectorized::IFunctionBase, (__gnu_cxx::_Lock_policy)2>&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1245:2
19:58:49 #4 0x56507fb89acd in std::shared_ptr<doris::vectorized::IFunctionBase>::operator=(std::shared_ptr<doris::vectorized::IFunctionBase>&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr.h:385:27
19:58:49 #5 0x56507fb89acd in doris::vectorized::FunctionCoalesce::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long) const /root/doris/be/src/vec/functions/function_coalesce.cpp:89:30
19:58:49 #6 0x56507958edfe in doris::vectorized::DefaultExecutable::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long) const /root/doris/be/src/vec/functions/function.h:463:26
19:58:49 #7 0x56507dbcbf5c in doris::vectorized::PreparedFunctionImpl::_execute_skipped_constant_deal(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp
19:58:49 #8 0x56507dbc65e3 in doris::vectorized::PreparedFunctionImpl::execute_without_low_cardinality_columns(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp:245:12
19:58:49 #9 0x56507dbc683b in doris::vectorized::PreparedFunctionImpl::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp:251:12
19:58:49 #10 0x56507958c81c in doris::vectorized::IFunctionBase::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.h:194:19
19:58:49 #11 0x565079578bee in doris::vectorized::VectorizedFnCall::_do_execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*, std::vector<unsigned long, std::allocator<unsigned long> >&) /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:188:5
19:58:49 #12 0x56507957a3e4 in doris::vectorized::VectorizedFnCall::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:203:12
19:58:49 #13 0x56507964994a in doris::vectorized::VExprContext::execute(doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vexpr_context.cpp:55:5
19:58:49 #14 0x56508f8d7496 in doris::pipeline::OperatorXBase::do_projections(doris::RuntimeState*, doris::vectorized::Block*, doris::vectorized::Block*) const /root/doris/be/src/pipeline/exec/operator.cpp:294:13
19:58:49 #15 0x56508f8d96ba in doris::pipeline::OperatorXBase::get_block_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*) /root/doris/be/src/pipeline/exec/operator.cpp:335:18
19:58:49 #16 0x565093c2560b in doris::pipeline::PipelineTask::execute(bool*) /root/doris/be/src/pipeline/pipeline_task.cpp:374:13
```
2024-10-29 14:26:47 +08:00
d0e7371077
[Test](bloom filter) add retry and retry timeout for bloom filter test case #42609 ( #42672 )
...
cherry pick from #42609
2024-10-29 13:33:02 +08:00
d5c43c51b5
[branch-2.1](fe) Avoid interrupt daemon thread and use proper polling interval ( #42210 ) ( #42646 )
...
pick https://github.com/apache/doris/pull/42210
2024-10-29 10:10:26 +08:00
78a8d12f9d
[Test](build index) enhance build index case for finished state #42630 ( #42651 )
...
cherry pick from #42630
2024-10-29 06:53:10 +08:00
73621c480a
[fix](log) fix group commit warn log ( #42403 ) ( #42575 )
...
pick https://github.com/apache/doris/pull/42403
2024-10-28 21:04:29 +08:00
f80c3d827a
[Fix](Branch-2.1) fix not null variant will lost null bitmap ( #42559 )
...
part of #39022
2024-10-28 20:59:47 +08:00
0baca54200
[fix](Nereids) offset do more than once when have shuffle after limit ( #42576 ) ( #42577 )
...
pick from master #42576
intro by #39316 . it want to fix a problem intro by #36699 . but forgot to
remove all wrong code in #36699 .
after #39316 , we should not set offset on exchange, when the exchange is
on the top of a limit with offset.
2024-10-28 19:48:13 +08:00
d319dafb5c
[pick](branch-2.1) pick #41123 ( #42541 )
...
cgroup memory usage should be refreshed frequently.
2024-10-28 19:21:19 +08:00
dfb5a6fdc7
[regression-test](prepared_stmt_p0) use master ip for prepared url ( #42521 ) ( #42567 )
...
(#42521 )
2024-10-28 17:47:55 +08:00
85674814eb
[fix](query-forward) Fix forward query exception or stuck or potential query result loss ( #41303 ) ( #42369 )
...
## Proposed changes
1. Fix forward query exception if no status code is set in master
execution. EOF may result in this status.
2. Fix forward query stuck due to no result packet sent to mysql
channel. Should use result packets from master.
3. Fix potential forward query result loss if follower can read status
change during query process. Should judge by the status once before
execution.
4. Add assertion for regression test.
2024-10-28 17:39:57 +08:00
59b6d7d472
[fix](cases)fix ip case2.1 with sync after streamload ( #42543 )
...
## Proposed changes
fix ip case2.1 with sync after streamload
Issue Number: close #xxx
<!--Describe your changes.-->
2024-10-28 14:54:36 +08:00
2c79d65433
[fix](case) fix jsonb case ( #42536 )
...
## Proposed changes
fix case for jsonb
Issue Number: close #xxx
<!--Describe your changes.-->
2024-10-28 14:02:56 +08:00
0e63133c80
[Chore](job) Provides configuration of job execution queue size ( #42253 ) ( #42530 )
...
When dealing with a large number of tasks, the default execution queue
size is 1024. This can lead to tasks being dropped if the queue becomes
full.
eg
`dispatch instant task failed, job id is xxx`
To address this, you can add the parameters `insert_task_queue_size` and
`mtmv_task_queue_size` in the `fe.conf` configuration file. These
parameters must be set to a power of 2.
**Keep in mind, increasing this value is recommended only when thread
resources are limited; otherwise, you should consider increasing the
number of task execution threads.**
(cherry picked from commit f9ea8f8229e9f5514c1773bd25c3cc11985c63fb)
## Proposed changes
Issue Number: #42253
<!--Describe your changes.-->
2024-10-28 13:42:08 +08:00
c3878167a5
[fix](config) change segment_cache_fd_percentage and inverted_index_fd_number_limit_percent default value #42224 ( #42423 )
2024-10-28 09:24:59 +08:00
4b5155453d
[fix](regression) fix unstable regression case: test_cte_filter_pushdown ( #42510 )
...
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
2024-10-27 20:59:31 +08:00
2bb29d7f36
[fix](test) fix move memtable injection test may cause other case stuck #40356 ( #42508 )
...
cherry pick from #40356
---------
Co-authored-by: hui lai <1353307710@qq.com >
2024-10-26 23:11:45 +08:00
1c9a798b68
[fix](case) fix insert_group_commit_into ( #42397 )
2024-10-25 22:15:38 +08:00
aa0053347f
[fix](crash) be crash on ~LRUFileCache ( #42498 )
...
come from: https://github.com/apache/doris/pull/39036
Crash stack:
```
(gdb) bt
#0 0x00007ff5c8c6c387 in raise () from /lib64/libc.so.6
#1 0x00007ff5c8c6da78 in abort () from /lib64/libc.so.6
#2 0x0000561eb0a5e38a in __gnu_cxx::__verbose_terminate_handler () at ../../../../libstdc++-v3/libsupc++/vterminate.cc:95
#3 0x0000561eb0a5caf6 in __cxxabiv1::__terminate (handler=<optimized out>) at ../../../../libstdc++-v3/libsupc++/eh_terminate.cc:48
#4 0x0000561eb0a5cb61 in std::terminate () at ../../../../libstdc++-v3/libsupc++/eh_terminate.cc:58
#5 0x0000561ea4028552 in doris::io::LRUFileCache::~LRUFileCache (this=0x7ff540fed000) at /root/be/src/io/cache/block/block_lru_file_cache.h:62
#6 0x0000561ea402857e in doris::io::LRUFileCache::~LRUFileCache (this=0x33688) at /root/be/src/io/cache/block/block_lru_file_cache.h:54
#7 0x0000561ea4251cd2 in std::default_delete<doris::io::IFileCache>::operator() (this=0x7ff54101d000, __ptr=0x33688)
at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:85
#8 std::unique_ptr<doris::io::IFileCache, std::default_delete<doris::io::IFileCache> >::~unique_ptr (this=0x7ff54101d000)
at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:361
#9 std::destroy_at<std::unique_ptr<doris::io::IFileCache, std::default_delete<doris::io::IFileCache> > > (__location=0x7ff54101d000)
at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88
#10 std::_Destroy<std::unique_ptr<doris::io::IFileCache, std::default_delete<doris::io::IFileCache> > > (__pointer=0x7ff54101d000)
at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:138
#11 std::_Destroy_aux<false>::__destroy<std::unique_ptr<doris::io::IFileCache, std::default_delete<doris::io::IFileCache> >*> (__first=0x7ff54101d000,
__last=0x7ff54101d008) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:152
#12 std::_Destroy<std::unique_ptr<doris::io::IFileCache, std::default_delete<doris::io::IFileCache> >*> (__first=<optimized out>, __last=0x7ff54101d008)
at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:184
#13 std::_Destroy<std::unique_ptr<doris::io::IFileCache, std::default_delete<doris::io::IFileCache> >*, std::unique_ptr<doris::io::IFileCache, std::default_delete<doris::io::IFileCache> > > (__first=<optimized out>, __last=0x7ff54101d008)
at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:746
#14 std::vector<std::unique_ptr<doris::io::IFileCache, std::default_delete<doris::io::IFileCache> >, std::allocator<std::unique_ptr<doris::io::IFileCache, std::default_delete<doris::io::IFileCache> > > >::~vector (this=0x7ff596a24ba8)
at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:680
#15 doris::io::FileCacheFactory::~FileCacheFactory (this=0x7ff596a24b80) at /root/be/src/io/cache/block/block_file_cache_factory.h:42
#16 doris::ExecEnv::destroy (this=0x561eb130b800 <doris::ExecEnv::GetInstance()::s_exec_env>) at /root/be/src/runtime/exec_env_init.cpp:651
#17 0x0000561ea35c3fe3 in main (argc=<optimized out>, argv=<optimized out>) at /root/be/src/service/doris_main.cpp:628
```
2024-10-25 20:42:20 +08:00
9eef393e2a
[pick]support cgroup v2 ( #42465 )
...
## Proposed changes
pick #39991 #39374 #36663
2024-10-25 20:13:27 +08:00
4a62d9e44b
Revert "[2.1][improvement](jdbc catalog) Add catalog property to enable jdbc connection pool" ( #42481 )
...
Reverts apache/doris#42255
We have found that after closing the connection pool, there will be
class loading problems and connection release problems for some data
sources. We will remove this function first and re-add it after solving
and testing it completely.
2024-10-25 19:37:36 +08:00
83b81c129f
[fix](move-memtable) fix auto partition load issue caused by #42039 ( #42484 ) ( #42486 )
...
backport #42484
2024-10-25 18:27:00 +08:00
859332c918
[improve](common) Add synchronized to avoid concurrent modification #42384 ( #42454 )
...
cherry pick from #42384
2024-10-25 16:33:53 +08:00
b88d4db6e7
[fix](auth)Fix use encryptkey should check auth ( #41791 ) ( #42105 )
...
pick from master #41791
2024-10-25 14:32:42 +08:00
120bf28d1e
[fix](statistics)Skip shadow index while analyzing a table. ( #42201 ) ( #42414 )
...
backport: https://github.com/apache/doris/pull/42201
2024-10-25 13:44:42 +08:00
0700449e8b
[fix](schema change) fix delete predicate incorrect comparisons result with empty strings during schema change ( #41064 ) ( #42427 )
...
```
insert into t1 (k1, k2, v1) value(1, '', 2);
delete form t1 where k1 = 1 and k2 = '';
alter table modify column v1 string
select * from t1 // expect 0 rows in return, but get 1 row.
```
2024-10-25 01:05:55 +08:00
972c37de95
[opt](scanner profile) Rename some filed name to keep consistent with audit log. #41993 ( #42137 )
...
cherry pick from #41993
2024-10-25 01:05:13 +08:00
2984a1b6c8
[improve](stream load) set NumberLoadedRows to zero when stream load transaction failed ( #41946 ) ( #42265 )
...
pick (#41946 )
Set NumberLoadedRows to zero when stream load failed.
before:
```
stream load result: {
"TxnId": 8589,
"Label": "c8e7c4fe-56b2-4e3b-b4cc-4f2a94cdd003",
"Comment": "",
"TwoPhaseCommit": "false",
"Status": "Fail",
"Message": "[DATA_QUALITY_ERROR]too many filtered rows",
"NumberTotalRows": 3,
"NumberLoadedRows": 1,
"NumberFilteredRows": 2,
"NumberUnselectedRows": 0,
"LoadBytes": 77,
"LoadTimeMs": 78,
"BeginTxnTimeMs": 0,
"StreamLoadPutTimeMs": 4,
"ReadDataTimeMs": 0,
"WriteDataTimeMs": 72,
"ReceiveDataTimeMs": 7,
"CommitAndPublishTimeMs": 0,
"ErrorURL": "XXX"
}
```
now:
```
stream load result: {
"TxnId": 8589,
"Label": "c8e7c4fe-56b2-4e3b-b4cc-4f2a94cdd003",
"Comment": "",
"TwoPhaseCommit": "false",
"Status": "Fail",
"Message": "[DATA_QUALITY_ERROR]too many filtered rows",
"NumberTotalRows": 3,
"NumberLoadedRows": 0,
"NumberFilteredRows": 2,
"NumberUnselectedRows": 0,
"LoadBytes": 77,
"LoadTimeMs": 78,
"BeginTxnTimeMs": 0,
"StreamLoadPutTimeMs": 4,
"ReadDataTimeMs": 0,
"WriteDataTimeMs": 72,
"ReceiveDataTimeMs": 7,
"CommitAndPublishTimeMs": 0,
"ErrorURL": "XXX"
}
```
2024-10-25 01:01:46 +08:00
db9c74c38f
[improvement](statistics)Drop expired external stats only when the catalog is dropped. ( #42244 ) ( #42410 )
...
backport: https://github.com/apache/doris/pull/42244
2024-10-24 23:47:53 +08:00
e6bc8ab79e
[Opt](config) adjust segment cache ( #42308 ) ( #42431 )
...
#42308
2024-10-24 23:05:36 +08:00
ffffbec49c
[opt](function) Avoid create_column inside cast_column function ( #41775 ) ( #42432 )
...
#41775
Co-authored-by: Mryange <59914473+Mryange@users.noreply.github.com >
2024-10-24 23:05:02 +08:00
f528a8aa06
[Opt](log) change frequent INFO log to DEBUG log in ColumnObject ( #42336 ) ( #42434 )
...
(#42336 )
2024-10-24 23:03:58 +08:00
dba6296d3e
[improvement](statistics)Skip auto analyze when table row count is not fully reported. (return -1) ( #42209 ) ( #42407 )
...
backport: https://github.com/apache/doris/pull/42209
2024-10-24 19:40:03 +08:00
2f8d63f052
[fix](load) Fix potential data loss during disk migration #42296 ( #42386 )
...
cherry pick from #42296
2024-10-24 17:19:38 +08:00
7e710701ce
[enhancement](bakcup) throw detailed msg for backup or restore ( #42288 ) ( #42312 )
...
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
2024-10-24 16:56:04 +08:00
d209c16d81
[fix](schema-change) fix the bug of alter column nullable when double writing ( #41737 ) ( #42352 )
...
pick master #41737
## problem
CREATE TABLE t (
`k1` VARCHAR(30) NOT NULL,
`v1` INT NOT NULL
)
alter table t modify column `v1` INT NULL
insert into value ('1', 2), ('1', 3);
core dump
## reason
Schema change leads to double writing, during double writing, the two
schemas and slots are as follows
```
old tablet schema
k1 varchar not null
v1 int not null
```
```
new tablet scheam
k1 varchar not null
v1 int null
```
```
slot
k1 varchar not null
v1 int not null
v1 int null
```
During the double writing process, when selecting slots through the
schema, only the column names and types were compared, without comparing
the nullable attributes, which led to the selection of the wrong slot.
Since the slot determines the nullable attribute of the block, the
nullable attribute of the columns in the block is different from that of
the columns in the schema, resulting in a core dump.
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
2024-10-24 16:49:46 +08:00
