database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
20,576 Commits 1 Branch 0 Tags
ae88d032db3f1066cc4e40e793aaa79674f86380
Commit Graph

8298 Commits

Author SHA1 Message Date
amory
e43d255815 [fix](array) fix add_array_null_value for inverted index_builder (#42899) 
## Proposed changes
Pick #42638 to doris branch 2.1
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-10-30 19:16:59 +08:00
Jerry Hu
848234bd92 [fix](exchange) query blocked by exchange reach limit (#42853) 
## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-10-30 10:13:26 +08:00
TengJianPing
42daf76a05 [fix](coalesce) fix 'heap-use-after-free' of function coalesce (#42666) (#42667) 
## Proposed changes
BP #42666

Should not modify state of vexpr when executing the expr, since it may
be executed in parallel.

Issue introduced by https://github.com/apache/doris/pull/42136

```
==75397==ERROR: AddressSanitizer: heap-use-after-free on address 0x60700294b060 at pc 0x56507fb8a370 bp 0x7fc861cd7050 sp 0x7fc861cd7048
19:58:49   READ of size 8 at 0x60700294b060 thread T1463 (Pipe_normal [wo)
19:58:49       #0 0x56507fb8a36f in doris::vectorized::FunctionCoalesce::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long) const /root/doris/be/src/vec/functions/function_coalesce.cpp:163:13
19:58:49       #1 0x56507958edfe in doris::vectorized::DefaultExecutable::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long) const /root/doris/be/src/vec/functions/function.h:463:26
19:58:49       #2 0x56507dbcbf5c in doris::vectorized::PreparedFunctionImpl::_execute_skipped_constant_deal(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp
19:58:49       #3 0x56507dbc65e3 in doris::vectorized::PreparedFunctionImpl::execute_without_low_cardinality_columns(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp:245:12
19:58:49       #4 0x56507dbc683b in doris::vectorized::PreparedFunctionImpl::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp:251:12
19:58:49       #5 0x56507958c81c in doris::vectorized::IFunctionBase::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.h:194:19
19:58:49       #6 0x565079578bee in doris::vectorized::VectorizedFnCall::_do_execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*, std::vector<unsigned long, std::allocator<unsigned long> >&) /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:188:5
19:58:49       #7 0x56507957a3e4 in doris::vectorized::VectorizedFnCall::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:203:12
19:58:49       #8 0x56507964994a in doris::vectorized::VExprContext::execute(doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vexpr_context.cpp:55:5
19:58:49       #9 0x56508f8d7496 in doris::pipeline::OperatorXBase::do_projections(doris::RuntimeState*, doris::vectorized::Block*, doris::vectorized::Block*) const /root/doris/be/src/pipeline/exec/operator.cpp:294:13
19:58:49       #10 0x56508f8d96ba in doris::pipeline::OperatorXBase::get_block_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*) /root/doris/be/src/pipeline/exec/operator.cpp:335:18
19:58:49       #11 0x565093c2560b in doris::pipeline::PipelineTask::execute(bool*) /root/doris/be/src/pipeline/pipeline_task.cpp:374:13
19:58:49       #12 0x565093c6a164 in doris::pipeline::TaskScheduler::_do_work(int) /root/doris/be/src/pipeline/task_scheduler.cpp:138:9
19:58:49       #13 0x565062529d1d in doris::ThreadPool::dispatch_thread() /root/doris/be/src/util/threadpool.cpp:543:24
19:58:49       #14 0x5650625021ce in std::function<void ()>::operator()() const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9
19:58:49       #15 0x5650625021ce in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:498:5
19:58:49       #16 0x7fca1844a608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8
19:58:49       #17 0x7fca186f7132 in __clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95
19:58:49   
19:58:49   0x60700294b060 is located 16 bytes inside of 80-byte region [0x60700294b050,0x60700294b0a0)
19:58:49   freed by thread T1462 (Pipe_normal [wo) here:
19:58:49       #0 0x56505dd2f80d in operator delete(void*) (/mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be+0x2f79e80d) (BuildId: 60cc2a41212f4e75)
19:58:49       #1 0x565079596a6f in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:702:11
19:58:49       #2 0x565079596a6f in std::__shared_ptr<doris::vectorized::IFunctionBase, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1149:31
19:58:49       #3 0x565079596a6f in std::__shared_ptr<doris::vectorized::IFunctionBase, (__gnu_cxx::_Lock_policy)2>::operator=(std::__shared_ptr<doris::vectorized::IFunctionBase, (__gnu_cxx::_Lock_policy)2>&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1245:2
19:58:49       #4 0x56507fb89acd in std::shared_ptr<doris::vectorized::IFunctionBase>::operator=(std::shared_ptr<doris::vectorized::IFunctionBase>&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr.h:385:27
19:58:49       #5 0x56507fb89acd in doris::vectorized::FunctionCoalesce::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long) const /root/doris/be/src/vec/functions/function_coalesce.cpp:89:30
19:58:49       #6 0x56507958edfe in doris::vectorized::DefaultExecutable::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long) const /root/doris/be/src/vec/functions/function.h:463:26
19:58:49       #7 0x56507dbcbf5c in doris::vectorized::PreparedFunctionImpl::_execute_skipped_constant_deal(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp
19:58:49       #8 0x56507dbc65e3 in doris::vectorized::PreparedFunctionImpl::execute_without_low_cardinality_columns(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp:245:12
19:58:49       #9 0x56507dbc683b in doris::vectorized::PreparedFunctionImpl::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp:251:12
19:58:49       #10 0x56507958c81c in doris::vectorized::IFunctionBase::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.h:194:19
19:58:49       #11 0x565079578bee in doris::vectorized::VectorizedFnCall::_do_execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*, std::vector<unsigned long, std::allocator<unsigned long> >&) /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:188:5
19:58:49       #12 0x56507957a3e4 in doris::vectorized::VectorizedFnCall::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:203:12
19:58:49       #13 0x56507964994a in doris::vectorized::VExprContext::execute(doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vexpr_context.cpp:55:5
19:58:49       #14 0x56508f8d7496 in doris::pipeline::OperatorXBase::do_projections(doris::RuntimeState*, doris::vectorized::Block*, doris::vectorized::Block*) const /root/doris/be/src/pipeline/exec/operator.cpp:294:13
19:58:49       #15 0x56508f8d96ba in doris::pipeline::OperatorXBase::get_block_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*) /root/doris/be/src/pipeline/exec/operator.cpp:335:18
19:58:49       #16 0x565093c2560b in doris::pipeline::PipelineTask::execute(bool*) /root/doris/be/src/pipeline/pipeline_task.cpp:374:13
```
 2024-10-29 14:26:47 +08:00
lihangyu
f80c3d827a [Fix](Branch-2.1) fix not null variant will lost null bitmap (#42559) 
part of #39022
 2024-10-28 20:59:47 +08:00
Xinyi Zou
d319dafb5c [pick](branch-2.1) pick #41123 (#42541) 
cgroup memory usage should be refreshed frequently.
 2024-10-28 19:21:19 +08:00
Kang
c3878167a5 [fix](config) change segment_cache_fd_percentage and inverted_index_fd_number_limit_percent default value #42224 (#42423) 2024-10-28 09:24:59 +08:00
Xin Liao
2bb29d7f36 [fix](test) fix move memtable injection test may cause other case stuck #40356 (#42508) 
cherry pick from #40356

---------

Co-authored-by: hui lai <1353307710@qq.com>
 2024-10-26 23:11:45 +08:00
camby
aa0053347f [fix](crash) be crash on ~LRUFileCache (#42498) 
come from: https://github.com/apache/doris/pull/39036

Crash stack:
```
(gdb) bt
#0  0x00007ff5c8c6c387 in raise () from /lib64/libc.so.6
#1  0x00007ff5c8c6da78 in abort () from /lib64/libc.so.6
#2  0x0000561eb0a5e38a in __gnu_cxx::__verbose_terminate_handler () at ../../../../libstdc++-v3/libsupc++/vterminate.cc:95
#3  0x0000561eb0a5caf6 in __cxxabiv1::__terminate (handler=<optimized out>) at ../../../../libstdc++-v3/libsupc++/eh_terminate.cc:48
#4  0x0000561eb0a5cb61 in std::terminate () at ../../../../libstdc++-v3/libsupc++/eh_terminate.cc:58
#5  0x0000561ea4028552 in doris::io::LRUFileCache::~LRUFileCache (this=0x7ff540fed000) at /root/be/src/io/cache/block/block_lru_file_cache.h:62
#6  0x0000561ea402857e in doris::io::LRUFileCache::~LRUFileCache (this=0x33688) at /root/be/src/io/cache/block/block_lru_file_cache.h:54
#7  0x0000561ea4251cd2 in std::default_delete<doris::io::IFileCache>::operator() (this=0x7ff54101d000, __ptr=0x33688)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:85
#8  std::unique_ptr<doris::io::IFileCache, std::default_delete<doris::io::IFileCache> >::~unique_ptr (this=0x7ff54101d000)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:361
#9  std::destroy_at<std::unique_ptr<doris::io::IFileCache, std::default_delete<doris::io::IFileCache> > > (__location=0x7ff54101d000)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88
#10 std::_Destroy<std::unique_ptr<doris::io::IFileCache, std::default_delete<doris::io::IFileCache> > > (__pointer=0x7ff54101d000)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:138
#11 std::_Destroy_aux<false>::__destroy<std::unique_ptr<doris::io::IFileCache, std::default_delete<doris::io::IFileCache> >*> (__first=0x7ff54101d000, 
    __last=0x7ff54101d008) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:152
#12 std::_Destroy<std::unique_ptr<doris::io::IFileCache, std::default_delete<doris::io::IFileCache> >*> (__first=<optimized out>, __last=0x7ff54101d008)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:184
#13 std::_Destroy<std::unique_ptr<doris::io::IFileCache, std::default_delete<doris::io::IFileCache> >*, std::unique_ptr<doris::io::IFileCache, std::default_delete<doris::io::IFileCache> > > (__first=<optimized out>, __last=0x7ff54101d008)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:746
#14 std::vector<std::unique_ptr<doris::io::IFileCache, std::default_delete<doris::io::IFileCache> >, std::allocator<std::unique_ptr<doris::io::IFileCache, std::default_delete<doris::io::IFileCache> > > >::~vector (this=0x7ff596a24ba8)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:680
#15 doris::io::FileCacheFactory::~FileCacheFactory (this=0x7ff596a24b80) at /root/be/src/io/cache/block/block_file_cache_factory.h:42
#16 doris::ExecEnv::destroy (this=0x561eb130b800 <doris::ExecEnv::GetInstance()::s_exec_env>) at /root/be/src/runtime/exec_env_init.cpp:651
#17 0x0000561ea35c3fe3 in main (argc=<optimized out>, argv=<optimized out>) at /root/be/src/service/doris_main.cpp:628
```
 2024-10-25 20:42:20 +08:00
wangbo
9eef393e2a [pick]support cgroup v2 (#42465) 
## Proposed changes

pick #39991   #39374  #36663
 2024-10-25 20:13:27 +08:00
zy-kkk
4a62d9e44b Revert "[2.1][improvement](jdbc catalog) Add catalog property to enable jdbc connection pool" (#42481) 
Reverts apache/doris#42255

We have found that after closing the connection pool, there will be
class loading problems and connection release problems for some data
sources. We will remove this function first and re-add it after solving
and testing it completely.
 2024-10-25 19:37:36 +08:00
Kaijie Chen
83b81c129f [fix](move-memtable) fix auto partition load issue caused by #42039 (#42484) (#42486) 
backport #42484
 2024-10-25 18:27:00 +08:00
Luwei
0700449e8b [fix](schema change) fix delete predicate incorrect comparisons result with empty strings during schema change (#41064) (#42427) 
```
insert into t1 (k1,  k2,  v1) value(1, '', 2);
delete form t1 where k1 = 1 and k2 = '';
alter table modify column v1 string
select * from t1 //  expect 0 rows in return, but get 1 row.
```
 2024-10-25 01:05:55 +08:00
zhiqiang
972c37de95 [opt](scanner profile) Rename some filed name to keep consistent with audit log. #41993 (#42137) 
cherry pick from #41993
 2024-10-25 01:05:13 +08:00
hui lai
2984a1b6c8 [improve](stream load) set NumberLoadedRows to zero when stream load transaction failed (#41946) (#42265) 
pick (#41946)

Set NumberLoadedRows to zero when stream load failed.

before:
```
stream load result: {
    "TxnId": 8589,
    "Label": "c8e7c4fe-56b2-4e3b-b4cc-4f2a94cdd003",
    "Comment": "",
    "TwoPhaseCommit": "false",
    "Status": "Fail",
    "Message": "[DATA_QUALITY_ERROR]too many filtered rows",
    "NumberTotalRows": 3,
    "NumberLoadedRows": 1,
    "NumberFilteredRows": 2,
    "NumberUnselectedRows": 0,
    "LoadBytes": 77,
    "LoadTimeMs": 78,
    "BeginTxnTimeMs": 0,
    "StreamLoadPutTimeMs": 4,
    "ReadDataTimeMs": 0,
    "WriteDataTimeMs": 72,
    "ReceiveDataTimeMs": 7,
    "CommitAndPublishTimeMs": 0,
    "ErrorURL": "XXX"
}
```

now:
```
stream load result: {
    "TxnId": 8589,
    "Label": "c8e7c4fe-56b2-4e3b-b4cc-4f2a94cdd003",
    "Comment": "",
    "TwoPhaseCommit": "false",
    "Status": "Fail",
    "Message": "[DATA_QUALITY_ERROR]too many filtered rows",
    "NumberTotalRows": 3,
    "NumberLoadedRows": 0,
    "NumberFilteredRows": 2,
    "NumberUnselectedRows": 0,
    "LoadBytes": 77,
    "LoadTimeMs": 78,
    "BeginTxnTimeMs": 0,
    "StreamLoadPutTimeMs": 4,
    "ReadDataTimeMs": 0,
    "WriteDataTimeMs": 72,
    "ReceiveDataTimeMs": 7,
    "CommitAndPublishTimeMs": 0,
    "ErrorURL": "XXX"
}
```
 2024-10-25 01:01:46 +08:00
lihangyu
e6bc8ab79e [Opt](config) adjust segment cache (#42308) (#42431) 
#42308
 2024-10-24 23:05:36 +08:00
lihangyu
ffffbec49c [opt](function) Avoid create_column inside cast_column function (#41775) (#42432) 
#41775

Co-authored-by: Mryange <59914473+Mryange@users.noreply.github.com>
 2024-10-24 23:05:02 +08:00
lihangyu
f528a8aa06 [Opt](log) change frequent INFO log to DEBUG log in ColumnObject (#42336) (#42434) 
(#42336)
 2024-10-24 23:03:58 +08:00
Xin Liao
2f8d63f052 [fix](load) Fix potential data loss during disk migration #42296 (#42386) 
cherry pick from #42296
 2024-10-24 17:19:38 +08:00
Luwei
d209c16d81 [fix](schema-change) fix the bug of alter column nullable when double writing (#41737) (#42352) 
pick master #41737 

## problem 

CREATE TABLE t (
    `k1` VARCHAR(30) NOT NULL,
    `v1` INT NOT NULL
)

alter table t modify column `v1` INT NULL

insert into value ('1', 2), ('1', 3);

core dump

## reason

Schema change leads to double writing, during double writing, the two
schemas and slots are as follows
 
```
old tablet schema 
k1 varchar not null
v1 int not null
```
 
```
new tablet scheam
k1 varchar not null
v1 int null
```

```
slot
k1 varchar not null
v1 int not null
v1 int null
```

During the double writing process, when selecting slots through the
schema, only the column names and types were compared, without comparing
the nullable attributes, which led to the selection of the wrong slot.
Since the slot determines the nullable attribute of the block, the
nullable attribute of the columns in the block is different from that of
the columns in the schema, resulting in a core dump.

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-10-24 16:49:46 +08:00
Kaijie Chen
d891ed9de8 [opt](load) check -238 segment number limit earlier (#39045) (#41551) 
backport #39045

---------

Co-authored-by: Xin Liao <liaoxinbit@126.com>
 2024-10-24 11:17:27 +08:00
Mryange
cbdaaa62b2 [feature](function) hour/minute/second functions support time as an a… 41008 (#42232) 
…rgument. (#41008)

## Proposed changes

```
mysql> select cast(4562632 as time),hour(cast(4562632 as time)), minute(cast(4562632 as time)),second(cast(4562632 as time));
+-----------------------+-----------------------------+-------------------------------+-------------------------------+
| cast(4562632 as TIME) | hour(cast(4562632 as TIME)) | minute(cast(4562632 as TIME)) | second(cast(4562632 as TIME)) |
+-----------------------+-----------------------------+-------------------------------+-------------------------------+
| 456:26:32             |                         456 |                            26 |                            32 |
+-----------------------+-----------------------------+-------------------------------+-------------------------------+
```

<!--Describe your changes.-->

---------

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->

Co-authored-by: Dongyang Li <hello_stephen@qq.com>
 2024-10-24 11:09:36 +08:00
zhangstar333
a3510d3b73 [cherry-pick](branch-21)fix the exchange not skip offset rows (#42190) (#42330) 
cherry-pick from master (#42190)
 2024-10-24 11:08:17 +08:00
TengJianPing
64d0c55e1d [fix](decimal256) support decimal256 for many functions (#42136) (#42353) 
BP #42136
 2024-10-24 10:52:28 +08:00
camby
32b13b285c [fix](index compaction) fix fd leak and mem leak while index compaction (#41915) (#42171) 
pick #41915 to branch-2.1
 2024-10-24 10:40:10 +08:00
hui lai
0d76c16d14 [fix](routine load) reduce routine load task consume log (#42058) (#42268) 
pick (#42058)

There is too much routine load task log, five million logs were
generated in 10 minutes.

```
grep 'consumer meet partition eof' be.INFO.log.20240930-164533 | wc -l 
5369624
```
 2024-10-23 23:30:15 +08:00
Xin Liao
86fa6a37c7 [fix](load) Fix the issue of high-concurrency single-replica load getting stuck #42297 (#42317) 
cherry pick from #42297
 2024-10-23 23:28:05 +08:00
airborne12
ccff47e813 [Fix](inverted index) fix use after free when duplicate key in index dir when index file writer open index #42207 (#42301) 
cherry pick from #42207
 2024-10-23 16:45:25 +08:00
qiye
bc84734291 [opt](index compaction)Optimize logic of picking columns for index compaction(#42051) (#42287) 
## Proposed changes

bp #42051
 2024-10-23 15:21:23 +08:00
amory
e7395198d4 [fix](estimate) func call with not filter will estimate some statisti… (#42302) 
…cs (#41989)

before this pr: use ! or not in or condition when table has been
analyzed we will meet
```
SELECT
  count(1)
FROM
  table_30_un_pa_ke_pr_di4
where
  col_int_undef_signed_not_null < -128
  or not array_contains(col_array_bigint__undef_signed, col_int_undef_signed_not_null);

ERROR 1105 (HY000): errCode = 2, detailMessage = Not-predicate meet unexpected child:
  array_contains(col_array_bigint__undef_signed, cast(col_int_undef_signed_not_null as BIGINT))
```

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-10-23 14:33:41 +08:00
Rayner Chen
157d67e7ca [enhance](hive) Add regression-test cases for hive text ddl and hive text insert and fix reading null string bug #42200 (#42273) 
cherry pick from #42200

Co-authored-by: Socrates <suxiaogang223@icloud.com>
 2024-10-22 23:56:57 +08:00
zy-kkk
bde8e2d474 [2.1][improvement](jdbc catalog) Add catalog property to enable jdbc connection pool (#42255) 
pick (#41992)

We initially introduced jdbc connection pool to improve the connection
performance of jdbc catalog, but we always found that connection pool
would bring some unexpected errors, so we chose to add a catalog
property: `enable_connection_pool` to choose whether to enable the jdbc
connection pool of jdbc catalog, and the default false.However, the
created catalog will still open the connection pool when it is upgraded,
and only the newly created catalog will be false

And we conducted performance tests on this, the performance loss is
within the expected range.

- Enable connection pool: mysqlslap -uroot -h127.0.0.1 -P9030
--concurrency=1 --iterations=100 --query='SELECT * FROM mysql.test.test
limit 1;' --create-schema=mysql --delimiter=";" --verbose
Benchmark
        Average number of seconds to run all queries: 0.008 seconds
        Minimum number of seconds to run all queries: 0.004 seconds
        Maximum number of seconds to run all queries: 0.133 seconds
        Number of clients running queries: 1
        Average number of queries per client: 1

- Disable connection pool: mysqlslap -uroot -h127.0.0.1 -P9030
--concurrency=1 --iterations=100 --query='SELECT * FROM
mysql_no_pool.test.test limit 1;' --create-schema=mysql --delimiter=";"
--verbose
Benchmark
        Average number of seconds to run all queries: 0.054 seconds
        Minimum number of seconds to run all queries: 0.047 seconds
        Maximum number of seconds to run all queries: 0.184 seconds
        Number of clients running queries: 1
        Average number of queries per client: 1
 2024-10-22 23:28:28 +08:00
Kaijie Chen
25d7d0b255 [fix](move-memtable) abstract multi-streams to one logical stream (#42039) (#42250) 
backport #42039
 2024-10-22 20:26:42 +08:00
Socrates
38e529cd29 [cherry-pick](branch-2.1) support decimal256 for parquet reader (#42241) 
## Proposed changes
pick pr: https://github.com/apache/doris/pull/41526
 2024-10-22 19:42:09 +08:00
zzzxl
e2bdac39fb [fix] Implementing match_phrase_edge without index query method (#41658) (#42098) 
pick from #41658
 2024-10-22 18:44:21 +08:00
Xinyi Zou
6f2bac012a [pick](branch-2.1) pick #39398 #41754 #41770 (#42231) 
pick #39398 #41754 #41770
 2024-10-22 18:05:40 +08:00
zclllhhjj
7eec0f8fbb [branch-2.1](datetime) Fix date floor functions overflow (#35477) (#42238) 
pick https://github.com/apache/doris/pull/35477
 2024-10-22 15:54:53 +08:00
Gabriel
8877267930 [pipeline](API) Add a new API to find pipeline tasks by a specific qu… (#42233) 
…ery ID (#35563)

pick #35563
 2024-10-22 14:03:45 +08:00
Rayner Chen
47ff6f1300 [fix](OrcReader) fix the issue that orc_reader can not read DECIMAL(0,0) type of orc file #41795 (#42220) 
cherry pick from #41795

Co-authored-by: Tiewei Fang <43782773+BePPPower@users.noreply.github.com>
 2024-10-22 10:10:25 +08:00
zzzxl
1f8d685f26 [fix](inverted index) multi_match supports any, all, phrase. (#41663) (#42097) 
https://github.com/apache/doris/pull/41663
 2024-10-22 10:10:02 +08:00
Rayner Chen
e713b92321 [fix](multi-catalog) Disable string dictionary filtering when predicate express is not slot #42113 (#42222) 
cherry pick from #42113

Co-authored-by: Socrates <suxiaogang223@icloud.com>
 2024-10-22 09:43:29 +08:00
Rayner Chen
084434e25c [Test](tvf) add regression tests for testing orc reader #41606 #42188 (#42120) 
cherry pick from #42031 #42188

---------

Co-authored-by: Tiewei Fang <43782773+BePPPower@users.noreply.github.com>
Co-authored-by: TieweiFang <ftw2139@163.com>
 2024-10-21 21:31:18 +08:00
Socrates
a3c1657c4b [cherry-pick](branch-2.1) check end of file when reading page (#42159) 
## Proposed changes
pick pr: https://github.com/apache/doris/pull/41816
 2024-10-21 17:01:04 +08:00
Socrates
a32ad0b1f7 [cherry-pick](branch-2.1) support reading brotli compressed parquet file (#42162) 
pick pr: https://github.com/apache/doris/pull/41875
 2024-10-21 16:48:09 +08:00
Rayner Chen
b9e2738ee6 [Fix](orc-reader) Fix StringRef nullptr data by add checking string_values empty. #42061 (#42154) 
cherry pick from #42061

Co-authored-by: Qi Chen <kaka11.chen@gmail.com>
 2024-10-21 16:26:23 +08:00
yiguolei
dc438649d9 [bugfix](handshake) brpc handshake should not use light pool (#42115) (#42127) 
The light pool may be full. Handshake is used to check the connection
state of brpc. Should not be interfered by the thread pool logic.

---------
pick #42115

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->

Co-authored-by: yiguolei <yiguolei@gmail.com>
 2024-10-19 16:19:17 +08:00
zzzxl
d5fef266ec [fix](inverted index) Fix incorrect exception handling (#42094) 
https://github.com/apache/doris/pull/41874
 2024-10-19 10:45:32 +08:00
amory
5db44a1b91 [fix](arrays_overlap) support arrays overlap with inverted index (#42090) 
## Proposed changes
backport : https://github.com/apache/doris/pull/41286
https://github.com/apache/doris/pull/41495
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-10-18 22:08:39 +08:00
zzzxl
dde0bf92ce [fix](inverted index) Fix incorrect usage of regexp compile_err (#41944) (#42085) 
https://github.com/apache/doris/pull/41944
 2024-10-18 22:06:59 +08:00
zhangstar333
460ff02997 [cherry-pick](branch-21)fix date_floor function return wrong result (#41948) (#42065) 
## Proposed changes

cherry-pick from master https://github.com/apache/doris/pull/41948

<!--Describe your changes.-->
 2024-10-18 21:54:22 +08:00
zhiqiang
03136baacf [fix](scanner) Fix incorrect _max_thread_num in scanner context when many queries are running. #41273 (#42016) 
cherry pick from #41273
 2024-10-18 18:08:07 +08:00