Commit Graph

8634 Commits

Author SHA1 Message Date
7c23af5b00 branch-2.1: [fix](opt)fix create expr tree to avoid the dangling reference #49082 (#49834)
Cherry-picked from #49082

Co-authored-by: zhangm365 <mzhang2048@gmail.com>
2025-04-08 12:01:50 +08:00
aad189cf40 [feature](function) upper lower support utf8 input (#49756)
### What problem does this PR solve?
https://github.com/apache/doris/pull/49231
2025-04-07 12:00:31 +08:00
0ea87fba6f branch-2.1: [Bug][function] fix the string cast jsonb cause null map have not init value #49810 (#49817)
Cherry-picked from #49810

Co-authored-by: HappenLee <happenlee@selectdb.com>
2025-04-06 10:16:11 +08:00
c0bc16d88f [fix](function) wrong result of arrays_overlap (#49403) (#49707)
Pick #49403
If the two arrays have the same non-null elements, they are considered
overlapping, and the result is 1.
If the two arrays have no common non-null elements and either array
contains a null element, the result is null.
Otherwise, the result is 0.

```
select arrays_overlap([1, 2, 3], [1, null]);  -- result should be 1

select arrays_overlap([2, 3], [1, null]);  -- result should be null

select arrays_overlap([2, 3], [1]);   -- result should be 0
```

### What problem does this PR solve?
2025-04-04 20:58:01 +08:00
672829096e [chore](enhancement) remove atomic load shared ptr usage (#49781) 2025-04-03 17:57:12 +08:00
145e393d3d branch-2.1: [fix](function) check return type is nullptr in FunctionBasePtr::build #49737 (#49761) 2025-04-02 20:23:41 +08:00
cf847b8c71 [fix](load) return DataQualityError when filtered rows exceeds limit (#47617) (#49289)
backport #47617
2025-04-02 19:09:21 +08:00
997db43dda branch-2.1: [improve](thrift) Config thrift_max_message_size for THREAD_POOL and … #49677 (#49724)
Cherry-picked from #49677

Co-authored-by: walter <maochuan@selectdb.com>
2025-04-01 17:16:58 +08:00
cc0b2585ac branch-2.1: [function](date) Support date trunc function #49540 (#49661) 2025-03-29 20:21:33 +08:00
1259ee5088 branch-2.1: [Feature](function) support year of week #48870 (#49012) 2025-03-29 11:24:45 +08:00
22b8dc2289 branch-2.1: [Opt](bvar) Add bvar for txn tablet map #49567 (#49627)
Cherry-picked from #49567

Co-authored-by: bobhan1 <baohan@selectdb.com>
2025-03-29 10:27:07 +08:00
05ffc62594 branch-2.1: [Fix](compaction) Fix full compaction error when compaction size is too large #48958 (#49493)
Cherry-picked from #48958

Co-authored-by: abmdocrt <lianyukang@selectdb.com>
2025-03-29 09:01:36 +08:00
4a31fc4e09 [Bug](fix) fix the percentile func result do not equal the percentile array rewrite result (#49379)
cherry pick https://github.com/apache/doris/pull/49351
2025-03-29 08:56:24 +08:00
63fb74b480 branch-2.1: [Bug](udf) should delete local ref when return error msg #48609 (#49528)
Problem Summary:
cherry-pick from master (#48609)
2025-03-28 09:57:18 +08:00
60f6975850 branch-2.1-pick: [Fix](txn) Remove TabletTxnInfo if version exists when publish version (#49564) (#49590)
pick https://github.com/apache/doris/pull/49564
2025-03-28 09:56:41 +08:00
11f8b53016 [fix](runtime) Avoid merging results into one large result in BufferControlBlock (#49571) 2025-03-27 19:49:58 +08:00
088edffded [fix](column) remove string64 in check_chars_length (#49537) 2025-03-27 19:47:58 +08:00
017e5a42fb [branch-2.1] pick 49325 fix query statistics leak in BE (#49397)
pick #49325
2025-03-24 19:59:15 +08:00
ce49f37a5e branch-2.1: [fix](core) fix subreplace when inputting a large number of empty strings #49241 (#49303)
Cherry-picked from #49241

Co-authored-by: Mryange <yanxuecheng@selectdb.com>
2025-03-20 22:56:44 +08:00
3f9e989d05 branch-2.1: [fix](arrow-flight-sql) Fix fetch_arrow_flight_schema thread pool (#49291) 2025-03-20 17:07:37 +08:00
5ecfee56b5 [cherry-pick](branch-2.1) Pick "[Fix](compaction) Fix nullptr when executing cold compaction (#48329)" (#48886) 2025-03-20 16:49:54 +08:00
8f79742f7d branch-2.1: [fix](arrow) Fix Arrow serialization and deserialization of Date/Datetime/Array/Map/Struct/Bitmap/HLL/Decimal256 types (#49244)
### What problem does this PR solve?

pick #48944 [fix](arrow) Fix UT DataTypeSerDeArrowTest of
Array/Map/Struct/Bitmap/HLL/Decimal256 types
pick #48398  [fix](arrow) Fix UT DataTypeSerDeArrowTest of Date type
2025-03-20 09:57:04 +08:00
f771a422a9 branch-2.1: [fix](column) fix ColumnWithTypeAndName::get_nested use-after-free when input Const(Nullable) column #48288 (#49258) 2025-03-20 09:53:20 +08:00
e4d6460b3c branch-2.1-pick: [Fix](SC) Prevent the values of __DORIS_VERSION_COL__ be wrongly replaced by fake version when merging tmp rowset in sort SC #49193 (#49222)
pick https://github.com/apache/doris/pull/49193
2025-03-19 17:33:25 +08:00
3b61f840f4 [fix](function) Undefined behavior in parse_url (#49149) (#49226) 2025-03-19 17:32:47 +08:00
1822120c13 branch-2.1: [fix](multi table load) error URL does not depend on the value of _number_filtered_rows #49111 (#49131)
Cherry-picked from #49111

Co-authored-by: hui lai <laihui@selectdb.com>
2025-03-19 09:54:36 +08:00
Pxl
7355cfe8b6 [Chore](runtime-filter) remove wrong check and set disable when SyncSizeClosure meet eof status (#49186) 2025-03-18 18:16:51 +08:00
6430ff365d [Bug](partition) should not do reset for the partition_sorts (#49148)
### What problem does this PR solve?
Problem Summary:

the partition_sorts is unique_ptr,could release after destroy.
and when some extreme case like cancel, if source reset the sorter
early, the sink operator will coredump still use it.
2025-03-18 18:03:29 +08:00
16e348b189 [fix](array/map) Fix BE crash in lambda functions (#49139) 2025-03-18 11:51:38 +08:00
afb143c1e8 [improve](binlog) Add config to control whether enable persistent connection during ingesting (#49005)
Cherry-pick #48467, #48761
2025-03-15 10:45:53 +08:00
e5a2b0eea8 Revert "[cherry-pick](jsonb) add a check for jsonb value to avoid invalid jsonb value write into segment file " (#49058)
Reverts apache/doris#48729
temp revert this pr for
PartialUpdateInfo::_generate_default_values_for_missing_cids using empty
string , which will make this check fail.
2025-03-14 17:41:06 +08:00
ad6cf63a28 branch-2.1: [opt](inverted index) uniform profile naming convention #48826 (#48975)
Cherry-picked from #48826

Co-authored-by: zzzxl <yangsiyu@selectdb.com>
2025-03-14 14:04:46 +08:00
79595ad62f [fix](minor) Reorder incorrect logging (#49001) 2025-03-14 12:03:34 +08:00
c989ac9467 branch-2.1: [chore](binlog) add ingesting/downloading binlog latency metrics #48599 (#49002)
cherry pick from #48599
2025-03-14 11:22:23 +08:00
7d521ce288 branch-2.1: [fix](binlog) avoid adding acqurie_md5 param when enable_download_md5… #48573 (#49004)
cherry pick from #48573
2025-03-14 11:18:51 +08:00
ed2e1ac34a branch-2.1: [fix](variant) update least common type in ColumnObject::pop_back #48935 (#48979)
Cherry-picked from #48935

Co-authored-by: Sun Chenyang <sunchenyang@selectdb.com>
2025-03-13 17:41:17 +08:00
ef4101b52a branch-2.1: [fix](cold hot separation) Fix the issue that files on the remote storage are not deleted after triggering cold data compaction. (#48109) (#48446)
Auto pick #48109 branch 2.1
2025-03-12 15:37:00 +08:00
e455bceb91 [fix](function) fix error result when STR_TO_DATE input all space (#4… (#48920)
…8872)
https://github.com/apache/doris/pull/48872
before
```
mysql> select STR_TO_DATE ('  ', '%Y-%m-%d %H:%i:%s');
+-----------------------------------------+
| STR_TO_DATE ('  ', '%Y-%m-%d %H:%i:%s') |
+-----------------------------------------+
|                                         |
+-----------------------------------------+
```
now
```
mysql> select STR_TO_DATE ('  ', '%Y-%m-%d %H:%i:%s');
+-----------------------------------------+
| STR_TO_DATE ('  ', '%Y-%m-%d %H:%i:%s') |
+-----------------------------------------+
| NULL                                    |
+-----------------------------------------+
```

Problem Summary:

None

- Test <!-- At least one of them must be included. -->
    - [x] Regression test
    - [x] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change. - [ ] No code files have been
changed. - [ ] Other reason <!-- Add your reason? -->

- Behavior changed:
    - [x] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->

### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-03-11 19:30:38 +08:00
bc3e93a8a9 Revert "branch-2.1: [bug](auto partition) Fix be crash with single replica insert" (#48926)
Reverts apache/doris#48536

BE core after this  PR, revert it。
*** Check failure stack trace: ***
    @     0x564fa82e5606  google::LogMessage::SendToLog()
    @     0x564fa82e2050  google::LogMessage::Flush()
    @     0x564fa82e5e49  google::LogMessageFatal::~LogMessageFatal()
    @     0x564fa9dcb44a  (unknown)
@ 0x564fa8a19e05 google::protobuf::internal::LogMessage::Finish()
    @     0x564f9e68d49e  google::protobuf::Map<>::at<>()
    @     0x564f9e68b805  doris::TabletsChannel::_commit_txn()
    @     0x564f9e68b20b  doris::TabletsChannel::close()
    @     0x564f9e591fee  doris::LoadChannel::_handle_eos()
    @     0x564f9e591ca2  doris::LoadChannel::add_batch()
    @     0x564f9e58c300  doris::LoadChannelMgr::add_batch()
    @     0x564f9e702bb1  std::_Function_handler<>::_M_invoke()
    @     0x564f9e71d3db  doris::WorkThreadPool<>::work_thread()
    @     0x564fab0ad760  execute_native_thread_routine
    @     0x7efd907a8ac3  (unknown)
    @     0x7efd9083a850  (unknown)
    @              (nil)  (unknown)
2025-03-11 17:44:34 +08:00
1462dc50d4 branch-2.1: [bug](auto partition) Fix be crash with single replica insert (#48536)
### What problem does this PR solve?

pick:#48101
2025-03-11 12:09:58 +08:00
4dbf3eb12b branch-2.1-pick: [Fix](merge-on-write) should re-calculate delete bitmaps between segments if BE restart before publish (#48775) (#48873)
pick https://github.com/apache/doris/pull/48775
2025-03-11 12:05:45 +08:00
7ca00c114b [fix](load) add lock for runtime_state->tablet_commit_infos (#48709) (#48850)
backport #48709
2025-03-11 12:03:39 +08:00
22f1cd1f64 [Improvement](local exchange) Reuse memory in PassToOneExchanger (#48745)
### What problem does this PR solve?

pick #39031

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-03-10 09:49:26 +08:00
e957a2f94c [improve](ipv6) Enhance ipv6 type to accept uint128 strings in network byte order (#48799) 2025-03-09 00:22:42 +08:00
Pxl
98c782bedb [Chore](case) pick FragmentMgr::send_filter_size.return_eof/RuntimeFilterProducer::send_size.rpc_fail to 2.1 (#48817)
part of https://github.com/apache/doris/pull/48225
2025-03-08 16:21:22 +08:00
48e3a73118 [fix](cancel) Fix cancel failure (#48751)
### What problem does this PR solve?

If a query is canceled before fragment contexts are prepared, no
fragment context will be found. So we should set execution ready to
ensure tasks will not be blocked.
2025-03-07 11:10:58 +08:00
3f684f2899 branch-2.1:[fix] (inverted index) Fix UTF-8 4-byte truncation issue and add configuration to control correct term writing (#48657) (#48741)
Cherry-picked from #48657
2025-03-06 21:28:24 +08:00
7b2899a7ff [cherry-pick](jsonb) add a check for jsonb value to avoid invalid jsonb value write into segment file (#48729)
…ke select core (#48625)

fix invalid jsonb value write into segment file which make select core,
so we add a check for jsonb value when convert_to_olap which value will
be written into segment file
2025-03-06 15:50:35 +08:00
d61737f04b branch-2.1: [fix](function) fix the function elt #48701 (#48719)
Cherry-picked from #48701

Co-authored-by: Sun Chenyang <sunchenyang@selectdb.com>
2025-03-06 11:39:54 +08:00
c9a299e914 [fix](columns) fix bug found by UT and add regression test (#48554) (#48690)
### What problem does this PR solve?

Issue Number: close #xxx

Related PR:  Pick #48554

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [x] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [x] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-03-06 09:33:33 +08:00