database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
20,041 Commits 1 Branch 0 Tags
70eeda9f50f9c4d202dedd6940044b4cdd2b56f5
Commit Graph

805 Commits

Author SHA1 Message Date
zhangstar333
70eeda9f50 [cherry-pick](branch-21) fix encryption_function return wrong result at const column (#40201) (#40247) 
## Proposed changes
cherry-pick from master #40201
the string object have error of order, should be src, key,iv,mode
 2024-09-02 16:09:15 +08:00
zclllhhjj
163193b1d4 [branch-2.1](function) fix random_bytes return same data for multi rows (#39891) (#40137) 
pick https://github.com/apache/doris/pull/39891

Issue Number: close #xxx

before:
```sql
mysql [optest]>SELECT random_bytes(10) a FROM numbers("number" = "10");
+------------------------+
| a                      |
+------------------------+
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
+------------------------+
```

now:
```sql
mysql [optest]>SELECT random_bytes(10) a FROM numbers("number" = "10");
+------------------------+
| a                      |
+------------------------+
| 0xd82cf60825b29ef2a0fd |
| 0x6f8c808415bdbaa6d257 |
| 0x7c26b5214297a151c25c |
| 0x43f02c77293063900437 |
| 0x5e5727569dec5e24f96b |
| 0x434f20bf74d7759640b7 |
| 0x087ed96b739750c733a6 |
| 0xdf05f6d7ede4972eb846 |
| 0xcefab471912264b5c54f |
| 0x1bddc019409d1926aa10 |
+------------------------+
```

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-30 10:43:42 +08:00
amory
b052f46944 [fix](json-quote) fix json quote func for not find the func (#39931) (#40061) 
1.  fix function not found before this pr:
```
drop table if exists t003;
create table t003 (a bigint, b json not null) properties ("replication_num"="1");
insert into t003 values (1, '{"a":1,"b":2}');
select a, map_agg("k1", json_quote(b)) from t003 group by a;

[17:47]>  select a, map_agg("k1", json_quote(b)) from t003 group by a;
(1105, 'errCode = 2, detailMessage = (172.20.48.119)[INTERNAL_ERROR]Function json_quote get failed, expr is VectorizedFnCall[json_quote](arguments=(CAST b(JSONB) TO String),return=Nullable(String)) and return type is Nullable(String).')
``` 
after pr , we can carry on it 
2. fix a core if we use json_quote with table column
```
[WARNING!] /sys/kernel/mm/transparent_hugepage/enabled: [always] madvise never, Doris not recommend turning on THP, which may cause the BE process to use more memory and cannot be freed in time. Turn off THP: `echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/enabled`
start BE in local mode
doris_be: /mnt/disk1/wangqiannan/amory/doris/thirdparty/installed/include/rapidjson/writer.h:488：void rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<>>>::Prefix(Type) [OutputStream = rapidjson::GenericStringBuffer<rapidjson::UTF8<>>, SourceEncoding = rapidjson::UTF8<>, TargetEncoding = rapidjson::UTF8<>, StackAllocator = rapidjson::CrtAllocator, writeFlags = 0]: 假设 ‘!hasRoot_’ 失败。
*** Query id: b7b94dead55e4090-8c8e94f65ec9efd3 ***
*** is nereids: 1 ***
*** tablet id: 0 ***
*** Aborted at 1724744881 (unix time) try "date -d @1724744881" if you are using GNU date ***
*** Current BE git commitID: bd5844ea0d ***
*** SIGABRT unknown detail explain (@0x461003611e1) received by PID 3543521 (TID 3548858 OR 0x7f8a53c1a700) from PID 3543521; stack trace: ***
doris_be: /mnt/disk1/wangqiannan/amory/doris/thirdparty/installed/include/rapidjson/writer.h:488：void rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<>>>::Prefix(Type) [OutputStream = rapidjson::GenericStringBuffer<rapidjson::UTF8<>>, SourceEncoding = rapidjson::UTF8<>, TargetEncoding = rapidjson::UTF8<>, StackAllocator = rapidjson::CrtAllocator, writeFlags = 0]: 假设 ‘!hasRoot_’ 失败。
doris_be: /mnt/disk1/wangqiannan/amory/doris/thirdparty/installed/include/rapidjson/writer.h:488：void rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<>>>::Prefix(Type) [OutputStream = rapidjson::GenericStringBuffer<rapidjson::UTF8<>>, SourceEncoding = rapidjson::UTF8<>, TargetEncoding = rapidjson::UTF8<>, StackAllocator = rapidjson::CrtAllocator, writeFlags = 0]: 假设 ‘!hasRoot_’ 失败。
doris_be: /mnt/disk1/wangqiannan/amory/doris/thirdparty/installed/include/rapidjson/writer.h:488：void rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<>>>::Prefix(Type) [OutputStream = rapidjson::GenericStringBuffer<rapidjson::UTF8<>>, SourceEncoding = rapidjson::UTF8<>, TargetEncoding = rapidjson::UTF8<>, StackAllocator = rapidjson::CrtAllocator, writeFlags = 0]: 假设 ‘!hasRoot_’ 失败。
doris_be: /mnt/disk1/wangqiannan/amory/doris/thirdparty/installed/include/rapidjson/writer.h:488：void rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<>>>::Prefix(Type) [OutputStream = rapidjson::GenericStringBuffer<rapidjson::UTF8<>>, SourceEncoding = rapidjson::UTF8<>, TargetEncoding = rapidjson::UTF8<>, StackAllocator = rapidjson::CrtAllocator, writeFlags = 0]: 假设 ‘!hasRoot_’ 失败。
doris_be: /mnt/disk1/wangqiannan/amory/doris/thirdparty/installed/include/rapidjson/writer.h:488：void rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<>>>::Prefix(Type) [OutputStream = rapidjson::GenericStringBuffer<rapidjson::UTF8<>>, SourceEncoding = rapidjson::UTF8<>, TargetEncoding = rapidjson::UTF8<>, StackAllocator = rapidjson::CrtAllocator, writeFlags = 0]: 假设 ‘!hasRoot_’ 失败。
doris_be: /mnt/disk1/wangqiannan/amory/doris/thirdparty/installed/include/rapidjson/writer.h:488：void rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<>>>::Prefix(Type) [OutputStream = rapidjson::GenericStringBuffer<rapidjson::UTF8<>>, SourceEncoding = rapidjson::UTF8<>, TargetEncoding = rapidjson::UTF8<>, StackAllocator = rapidjson::CrtAllocator, writeFlags = 0]: 假设 ‘!hasRoot_’ 失败。
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /mnt/disk1/wangqiannan/amory/doris/be/src/common/signal_handler.h:421
 1# 0x00007F9429595B50 in /lib64/libc.so.6
 2# gsignal in /lib64/libc.so.6
 3# __GI_abort in /lib64/libc.so.6
 4# _nl_load_domain.cold.0 in /lib64/libc.so.6
 5# 0x00007F942958E426 in /lib64/libc.so.6
 6# rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<char>, rapidjson::CrtAllocator>, rapidjson::UTF8<char>, rapidjson::UTF8<char>, rapidjson::CrtAllocator, 0u>::Prefix(rapidjson::Type) at /mnt/disk1/wangqiannan/amory/doris/thirdparty/installed/include/rapidjson/writer.h:488
 7# rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<char>, rapidjson::CrtAllocator>, rapidjson::UTF8<char>, rapidjson::UTF8<char>, rapidjson::CrtAllocator, 0u>::String(char const*, unsigned int, bool) at /mnt/disk1/wangqiannan/amory/doris/thirdparty/installed/include/rapidjson/writer.h:206
 8# bool rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >::Accept<rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<char>, rapidjson::CrtAllocator>, rapidjson::UTF8<char>, rapidjson::UTF8<char>, rapidjson::CrtAllocator, 0u> >(rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<char>, rapidjson::CrtAllocator>, rapidjson::UTF8<char>, rapidjson::UTF8<char>, rapidjson::CrtAllocator, 0u>&) const at /mnt/disk1/wangqiannan/amory/doris/thirdparty/installed/include/rapidjson/document.h:1974
 9# doris::vectorized::FunctionJsonQuoteImpl::execute(std::vector<doris::vectorized::ColumnStr<unsigned int> const*, std::allocator<doris::vectorized::ColumnStr<unsigned int> const*> > const&, doris::vectorized::ColumnStr<unsigned int>&, unsigned long) at /mnt/disk1/wangqiannan/amory/doris/be/src/vec/functions/function_json.cpp:826
10# doris::vectorized::FunctionJson<doris::vectorized::FunctionJsonQuoteImpl>::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long) const at /mnt/disk1/wangqiannan/amory/doris/be/src/vec/functions/function_json.cpp:985
11# doris::vectorized::DefaultExecutable::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long) const at /mnt/disk1/wangqiannan/amory/doris/be/src/vec/functions/function.h:463
12# doris::vectorized::PreparedFunctionImpl::_execute_skipped_constant_deal(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) const at /mnt/disk1/wangqiannan/amory/doris/be/src/vec/functions/function.cpp:120
13# doris::vectorized::PreparedFunctionImpl::execute_without_low_cardinality_columns(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) const at /mnt/disk1/wangqiannan/amory/doris/be/src/vec/functions/function.cpp:245
```

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-28 21:31:08 +08:00
amory
fc9936d923 [fix](funcs) fix map struct construct funcs (#39973) 
## Proposed changes
backport: https://github.com/apache/doris/pull/39699
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-27 18:31:51 +08:00
camby
b7e0bfa1c2 [improve](function) opt aes_encrypt/decrypt function to handle const column (#37194) (#39954) 
pick #37194 to branch-2.1

---------

Co-authored-by: zhangstar333 <87313068+zhangstar333@users.noreply.github.com>
 2024-08-27 14:56:38 +08:00
amory
4cf769b39f [Improve](table-function) explode json array with json args (#39491) 2024-08-27 14:53:17 +08:00
zhangstar333
21bd4a4ac8 [bug](function)fix json_replace check return type error (#37014) (#39938) 
1. fix the return type dcheck error:
```
mysql [test]>select (json_replace(a, '$.fparam.nested_2', "qwe")) from json_table_2 limit 1;
ERROR 1105 (HY000): errCode = 2, detailMessage = (10.16.10.8)[INTERNAL_ERROR]Function json_replace get failed, expr is VectorizedFnCall[json_replace](arguments=a, String, String, String,return=Nullable(String)) and return type is Nullable(String).
```

2. improve the json_replace/json_insert/json_set function execute of not
convert const column, test about could faster 1s on 1000w table rows

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-27 08:03:48 +08:00
zclllhhjj
460605ae3c [branch-2.1] pick some prs (#39860) 
## Proposed changes

Issue Number: close #xxx

https://github.com/apache/doris/pull/38385 optimize parsing datetime
https://github.com/apache/doris/pull/38978 make stream load failure
message more clear and disable some error's stacktrace by default
https://github.com/apache/doris/pull/39255 fix random function coredump
https://github.com/apache/doris/pull/39324 fix function corr
inconsistency with doc
https://github.com/apache/doris/pull/39449 check auto partitoin nullity
when creating partition
https://github.com/apache/doris/pull/39695 make
DynamicPartitionScheduler immediately know interval's change
https://github.com/apache/doris/pull/39754 Add some partition expr check
on creating table
 2024-08-24 17:26:42 +08:00
lihangyu
a6f267c479 [pick](Variant) fix element_at should return nullable if result type is nullable (#39846) 
#39732
 2024-08-24 09:22:03 +08:00
Socrates
bb687bd69c [cherry-pick](branch-2.1) add function regexp_extract_or_null (#39561) 
# Proposed changes

pick https://github.com/apache/doris/pull/38296
 2024-08-21 09:14:58 +08:00
zzzxl
b0da8430bc [opt](inverted index) Optimize the usage of the multi_match function (#39472) 
## Proposed changes

https://github.com/apache/doris/pull/39193

<!--Describe your changes.-->
 2024-08-17 16:53:52 +08:00
amory
79aa079cc6 [fix](array-funcs) array min/max #39307 (#39484) 2024-08-17 10:56:44 +08:00
amory
7687f2c53a [fix](ip-funcs) fix ip inet6_aton funcs #39415 (#39513) 2024-08-17 10:56:06 +08:00
amory
226e01889c [fix](array_apply) pick array apply fix (#39328) 
## Proposed changes
backport: https://github.com/apache/doris/pull/39105
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-14 18:52:29 +08:00
camby
b976cbf14d [opt](log) avoid lots of json parse error logs (#39190) (#39246) 
pick https://github.com/apache/doris/pull/39190 to branch-2.1
 2024-08-13 17:01:10 +08:00
Sun Chenyang
60eeec3754 [fix] (inverted index) Fix match function without inverted index (#38989) (#39220) 
## Proposed changes

pick from #38989
 2024-08-13 10:55:54 +08:00
Jerry Hu
96260d97bd [fix](ub) undefined behavior in FixedContainer (#39191) (#39201) 
## Proposed changes

pick #39191
Undefined behavior occurs if there is a null value in the list.

```
/root/doris/be/src/vec/common/string_ref.h:271:54: runtime error: null pointer passed as argument 2, which is declared to never be null
/var/local/ldb-toolchain/bin/../usr/include/string.h:64:33: note: nonnull attribute specified here
#0 0x5616d072245d in doris::StringRef::eq(doris::StringRef const&) const /root/doris/be/src/vec/common/string_ref.h:271:41
#1 0x5616d072245d in doris::StringRef::operator==(doris::StringRef const&) const /root/doris/be/src/vec/common/string_ref.h:274:60
#2 0x5616d072245d in doris::FixedContainer::find(doris::StringRef const&) const /root/doris/be/src/exprs/hybrid_set.h:76:36
#3 0x5616d072245d in void doris::StringValueSet>::_find_batch(doris::vectorized::IColumn const&, unsigned long, doris::vectorized::PODArray, 16ul, 15ul> const*, doris::vectorized::PODArray, 16ul, 15ul>&) /root/doris/be/src/exprs/hybrid_set.h:688:63
#4 0x5616d0747857 in doris::vectorized::FunctionIn::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long) const /root/doris/be/src/vec/functions/in.h:170:21
#5 0x5616c741fa3a in doris::vectorized::DefaultExecutable::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long) const /root/doris/be/src/vec/functions/function.h:462:26
#6 0x5616cbb5b650 in doris::vectorized::PreparedFunctionImpl::_execute_skipped_constant_deal(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp
#7 0x5616cbb4e14e in doris::vectorized::PreparedFunctionImpl::execute_without_low_cardinality_columns(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp:244:12
#8 0x5616cbb4e3c2 in doris::vectorized::PreparedFunctionImpl::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.cpp:250:12
#9 0x5616c741cd68 in doris::vectorized::IFunctionBase::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector> const&, unsigned long, unsigned long, bool) const /root/doris/be/src/vec/functions/function.h:190:19
#10 0x5616c74cf712 in doris::vectorized::VInPredicate::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vin_predicate.cpp:130:5
#11 0x5616c740d5c0 in doris::vectorized::VectorizedFnCall::_do_execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*, std::vector>&) /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:183:9
#12 0x5616c740ecf5 in doris::vectorized::VectorizedFnCall::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:215:12
#13 0x5616c7462e24 in doris::vectorized::VCompoundPred::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vcompound_pred.h:127:38
#14 0x5616c74bccec in doris::vectorized::VExprContext::execute(doris::vectorized::Block*, int*) /root/doris/be/src/vec/exprs/vexpr_context.cpp:54:5
#15 0x5616c74c1dcc in doris::vectorized::VExprContext::execute_conjuncts(std::vector, std::allocator>> const&, std::vector, 16ul, 15ul>, std::allocator, 16ul, 15ul>>> const*, bool, doris::vectorized::Block*, doris::vectorized::PODArray, 16ul, 15ul>, bool) /root/doris/be/src/vec/exprs/vexpr_context.cpp:169:9
#16 0x5616c74c5108 in doris::vectorized::VExprContext::execute_conjuncts_and_filter_block(std::vector, std::allocator>> const&, doris::vectorized::Block*, std::vector>&, int, doris::vectorized::PODArray, 16ul, 15ul>&) /root/doris/be/src/vec/exprs/vexpr_context.cpp:322:5
#17 0x5616ad8a7f1a in doris::segment_v2::SegmentIterator::_execute_common_expr(unsigned short*, unsigned short&, doris::vectorized::Block*) /root/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2680:5
#18 0x5616ad89e86e in doris::segment_v2::SegmentIterator::_next_batch_internal(doris::vectorized::Block*) /root/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2582:25
#19 0x5616ad892f5c in doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*)::$_0::operator()() const /root/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2315:9
#20 0x5616ad892f5c in doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*) /root/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2314:19
#21 0x5616ad6dd9cc in doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::vectorized::Block*) /root/doris/be/src/olap/rowset/segment_v2/lazy_init_segment_iterator.h:44:33
#22 0x5616ad269d67 in doris::BetaRowsetReader::next_block(doris::vectorized::Block*) /root/doris/be/src/olap/rowset/beta_rowset_reader.cpp:380:29
#23 0x5616de6de110 in doris::vectorized::VCollectIterator::Level0Iterator::_refresh() /root/doris/be/src/vec/olap/vcollect_iterator.h
#24 0x5616de6c967f in doris::vectorized::VCollectIterator::Level0Iterator::refresh_current_row() /root/doris/be/src/vec/olap/vcollect_iterator.cpp:514:24
#25 0x5616de6ca8a6 in doris::vectorized::VCollectIterator::Level0Iterator::ensure_first_row_ref() /root/doris/be/src/vec/olap/vcollect_iterator.cpp:493:14
#26 0x5616de6d7008 in doris::vectorized::VCollectIterator::Level1Iterator::ensure_first_row_ref() /root/doris/be/src/vec/olap/vcollect_iterator.cpp:692:27
#27 0x5616de6bd200 in doris::vectorized::VCollectIterator::build_heap(std::vector, std::allocator>>&) /root/doris/be/src/vec/olap/vcollect_iterator.cpp:186:9
#28 0x5616de651b6c in doris::vectorized::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) /root/doris/be/src/vec/olap/block_reader.cpp:157:5
#29 0x5616de65526f in doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) /root/doris/be/src/vec/olap/block_reader.cpp:229:19
#30 0x5616e175a0f9 in doris::vectorized::NewOlapScanner::open(doris::RuntimeState*) /root/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:237:32
#31 0x5616c736ad34 in doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr, std::shared_ptr) /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:236:5
#32 0x5616c736f05e in doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()::operator()() const::'lambda'()::operator()() const /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:176:21
#33 0x5616c736f05e in doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()::operator()() const /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:175:31
#34 0x5616c736f05e in void std::_invoke_impl, std::shared_ptr)::$_1::operator()() const::'lambda'()&>(std::_invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
#35 0x5616c736f05e in std::enable_if, std::shared_ptr)::$1::operator()() const::'lambda'()&>, void>::type std::_invoke_r, std::shared_ptr)::$_1::operator()() const::'lambda'()&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2
#36 0x5616c736f05e in std::_Function_handler, std::shared_ptr)::$_1::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9
#37 0x5616aeed6a3b in doris::ThreadPool::dispatch_thread() /root/doris/be/src/util/threadpool.cpp:543:24
#38 0x5616aeeae4f7 in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:498:5
#39 0x7f7e663e3ac2 in start_thread nptl/pthread_create.c:442:8
#40 0x7f7e6647584f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /root/doris/be/src/vec/common/string_ref.h:271:54 in
```

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-12 10:02:07 +08:00
yangshijie
5f77f909d9 [cherry-pick](branch-2.1) Pick "[feature](function) support ip functions named ipv4_to_ipv6 and cut_ipv6" (#39058) 
## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
pick https://github.com/apache/doris/pull/36883 and
https://github.com/apache/doris/pull/35239
 2024-08-10 18:37:11 +08:00
zzzxl
5e1e725cee [feature](inverted index) Add multi_match function #37722 #38931 #39149 (#38877) 2024-08-10 15:20:08 +08:00
lihangyu
773008d6fa [Fix](Json) fix some cast issue (#38683) (#39025) 
#38683
 2024-08-07 22:05:43 +08:00
lihangyu
7550fbaff7 [Fix](Exception) throw exception in defer may result std::terminate (… (#39007) 
pick #38935
 2024-08-07 13:46:23 +08:00
zzzxl
8cb5aa64f4 [test](inverted index) add an Inverted Index Testing Switch (#38077) (#38947) 
https://github.com/apache/doris/pull/38077
 2024-08-07 11:25:36 +08:00
Uniqueyou
65154f8abe [branch-2.1] (doris-future) Support auto partition name function (#38853) 
cherry-pick https://github.com/apache/doris/pull/34258 to branch-2.1
 2024-08-05 16:04:24 +08:00
amory
2653087843 [pick](array-funcs)fix array with empty arg in be behavior (#38708) 
## Proposed changes
backport: https://github.com/apache/doris/pull/36845
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-05 09:08:28 +08:00
amory
556f0fc784 [pick](json-keys) support json_keys function (#38631) 
## Proposed changes
backport: https://github.com/apache/doris/pull/36411
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-02 19:10:00 +08:00
amory
9b07cd2069 [pick](json-serde)pick jsonb string deserialize with spec char (#38711) 
## Proposed changes
backport: https://github.com/apache/doris/pull/37176
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-02 13:37:41 +08:00
amory
1d982ada45 [pick](array-funcs)pick array func array_enumerate_uniq bugfix (#38721) 
## Proposed changes
backport: https://github.com/apache/doris/pull/38384
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-02 11:25:17 +08:00
amory
f5bc65989c [pick](array-range)improve array_range func for large param (#38707) 
## Proposed changes
backport: https://github.com/apache/doris/pull/38284
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-02 11:22:46 +08:00
amory
184b8cbbe4 [pick](json)fix jsonb deseriaze (#38630) 
## Proposed changes
backport: https://github.com/apache/doris/pull/37251
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-01 10:18:27 +08:00
airborne12
66ebf709ba [Fix](inverted index) fix fast execute for not_in expr #37745 (#38594) 
cherry pick from #37745
 2024-07-31 19:58:12 +08:00
lihangyu
73fc55b203 [Pick](Variant) fix some issue by RQG (#38336) 
#38318 
#38291
 2024-07-25 12:19:07 +08:00
zhangstar333
70cde39fe0 [cherry-pick](branch-21) fix conv function get wrong result as parse overflow (#38001) (#38309) 
## Proposed changes

cherry-pick from https://github.com/apache/doris/pull/38001

<!--Describe your changes.-->
 2024-07-25 12:06:46 +08:00
zhangstar333
e9052e2180 [cherry-pick](branch-21) fix mod function cause core dump (#37999) (#38308) 
## Proposed changes
cherry-pick from master https://github.com/apache/doris/pull/37999

<!--Describe your changes.-->

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-25 12:06:21 +08:00
Jerry Hu
d19b3a5cb6 [fix](function) the bucket number arg of width_bucket should be a positive integer value (#37892) (#38295) 
## Proposed changes

pick #37892
 2024-07-25 11:56:47 +08:00
zclllhhjj
21b3fc3d1e [branch-2.1](function) fix coredump for MULTI_MATCH_ANY (#37959) (#38314) 
pick https://github.com/apache/doris/pull/37959

[INVALID_ARGUMENT][E33] Compile regexp expression failed. got Embedded
start anchors not supported.. some expressions may be illegal
 2024-07-25 11:34:22 +08:00
zclllhhjj
79a6496bb6 [branch-2.1](function) fix wrong result when convert_tz is out of bound (#37358) (#38313) 
## Proposed changes

pick https://github.com/apache/doris/pull/37358

before:
```sql
mysql> select CONVERT_TZ(cast('0000-01-01 00:00:00.00001'  as DATETIMEV1), cast('Asia/Shanghai' as VARCHAR(65533)), cast('America/Los_Angeles' as VARCHAR(65533)));
+---------------------------------------------------------------------------------------------------------------------------------------------------+
| convert_tz(cast('0000-01-01 00:00:00.00001' as DATETIME), cast('Asia/Shanghai' as VARCHAR(65533)), cast('America/Los_Angeles' as VARCHAR(65533))) |
+---------------------------------------------------------------------------------------------------------------------------------------------------+
| q535-12-31 08:01:19                                                                                                                               |
+---------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.12 sec)
```
now:
```sql
mysql> select CONVERT_TZ(cast('0000-01-01 00:00:00.00001'  as DATETIMEV1), cast('Asia/Shanghai' as VARCHAR(65533)), cast('America/Los_Angeles' as VARCHAR(65533)));
+---------------------------------------------------------------------------------------------------------------------------------------------------+
| convert_tz(cast('0000-01-01 00:00:00.00001' as DATETIME), cast('Asia/Shanghai' as VARCHAR(65533)), cast('America/Los_Angeles' as VARCHAR(65533))) |
+---------------------------------------------------------------------------------------------------------------------------------------------------+
| NULL                                                                                                                                              |
+---------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.09 sec)
```
 2024-07-25 11:32:44 +08:00
amory
947d7594c8 [fix](func)fix array_with_const with larger than max_array_size (#38152) 
## Proposed changes
backport: https://github.com/apache/doris/pull/37495
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-20 00:19:00 +08:00
camby
de2272ce48 [fix](round) fix round decimal128 overflow (#37733) (#37963) 
cherry-pick #37733 to branch-2.1
 2024-07-18 23:50:23 +08:00
zzzxl
0e248e3594 [fix](inverted index) Corrected the issue of no_index_match failure caused by empty data #37947 (#38002) 2024-07-18 10:04:36 +08:00
Mryange
cc6ff12097 [opt](function) Optimize the trim function for single-char inputs (#3… (#37799) 
https://github.com/apache/doris/pull/36497

before
```
mysql [test]>select count(ltrim(str,"1")) from stringDb2;
+------------------------+
| count(ltrim(str, '1')) |
+------------------------+
|               64000000 |
+------------------------+
1 row in set (7.79 sec)
```

now
```
mysql [test]>select count(ltrim(str,"1")) from stringDb2;
+------------------------+
| count(ltrim(str, '1')) |
+------------------------+
|               64000000 |
+------------------------+
1 row in set (0.73 sec)
```

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-16 17:52:52 +08:00
zhiqiang
02716598d4 [Fix](sql function) memory overflow to the left of string address when do_money_format has small negative value #36226 (#37870) 
cherry pick from #36226

Co-authored-by: sparrow <38098988+biohazard4321@users.noreply.github.com>
 2024-07-16 15:04:42 +08:00
Mryange
8e42871228 [fix](in expr) fix error result when in expr has null value and lef… (#37800) 
https://github.com/apache/doris/pull/36024

## Proposed changes

```
create table t2 (id int, c1 int);
insert into t2 values(1, null);
 select 0 in (c1, null) from t2; -- should return null，but 1
```

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-16 14:04:35 +08:00
Pxl
d7e84b7ee3 [Enchancement](bitmap) optimize bitmap deserialize and remove some unused code (#37623) 
## Proposed changes
pick from #35789
 2024-07-16 11:21:54 +08:00
amory
3cb1d4e842 [feature](json)support explode_json_object func #36887 (#37378) 2024-07-16 10:59:11 +08:00
zhangstar333
1d49d386aa [cherry-pick](branch-21) remove the useless code in column vector (#34432) (#37827) 
cherry-pick from master https://github.com/apache/doris/pull/34432

Co-authored-by: HappenLee <happenlee@hotmail.com>
 2024-07-15 22:10:58 +08:00
zzzxl
8df2432e94 [fix](inverted index) implementation of match function without index #36471 (#36918) 2024-07-15 16:19:41 +08:00
camby
8360e3f6cf [fix](sleep) sleep with character const make be crash (#37681) (#37775) 
cherry-pick #37681 to branch-2.1
 2024-07-15 14:57:46 +08:00
Mryange
8de13c5cc8 [fix](function) error scale set in unix_timestamp (#36110) (#37619) 
## Proposed changes

```
mysql [test]>set DEBUG_SKIP_FOLD_CONSTANT = true;
Query OK, 0 rows affected (0.00 sec)

mysql [test]>select cast(unix_timestamp("2024-01-01",'yyyy-MM-dd') as bigint);
+------------------------------------------------------------+
| cast(unix_timestamp('2024-01-01', 'yyyy-MM-dd') as BIGINT) |
+------------------------------------------------------------+
|                                           1704038400000000 |
+------------------------------------------------------------+
```
now
```
mysql [test]>select cast(unix_timestamp("2024-01-01",'yyyy-MM-dd') as bigint);
+------------------------------------------------------------+
| cast(unix_timestamp('2024-01-01', 'yyyy-MM-dd') as BIGINT) |
+------------------------------------------------------------+
|                                                 1704038400 |
+------------------------------------------------------------+
1 row in set (0.01 sec)
```

The column does not have a scale set, but the cast uses the scale to
perform the cast.


<!--Describe your changes.-->

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-15 10:00:04 +08:00
lihangyu
217eac790b [pick](Variant) pick some refactor and fix #34925 #36317 #36201 #36793 (#37526) 2024-07-11 21:25:34 +08:00
camby
239bc1a7e0 [fix](compile) fix compile failed on MacOS due to ambiguous std::abs (#37136) 
cherry-pick #35125 to branch-2.1

Co-authored-by: morrySnow <101034200+morrySnow@users.noreply.github.com>
 2024-07-02 17:45:33 +08:00