Commit Graph

3585 Commits

Author SHA1 Message Date
3aad9e5f67 [opt](oceanbase docker)Use LTS docker image and print unhealthy docker logs (#46647) (#47349)
### What problem does this PR solve?

Problem Summary:
Oceanbase container sometimes start failed.
<img width="653" alt="image"

src="https://github.com/user-attachments/assets/d95c66cf-7e04-4179-a565-9b9dd8b87128"
/>

We do two things:
1. Print last 100 lines docker logs of unhealthy container for debugging
2. Upgrade Oceanbase docker image to the newest `4.2.1-lts`, since it is
7 months newer than `4.2.1`, more stable
2025-01-24 11:22:02 +08:00
4f9562650d Branch-2.1 [Fix](Variant) fix variant serialize to string (#47121) (#47147)
cherry-pick from #47121
2025-01-18 09:12:39 +08:00
9dffe5992e branch-2.1: [fix](json) fix parsing double in jsonb #46977 (#47065)
Cherry-picked from #46977

Co-authored-by: Sun Chenyang <sunchenyang@selectdb.com>
2025-01-18 00:25:13 +08:00
f2fe07a0b9 [fix](regression) join condition is missing in tpcds query24 (#47158) 2025-01-17 20:31:28 +08:00
adc9e9a161 [fix](function) Fix undefined behavior caused by bit_shift_left exceeding the 64-bit range. (#47080)
### What problem does this PR solve?
It is part of this PR's https://github.com/apache/doris/pull/35769
content.


```
SELECT a, b, bit_shift_left(nullable(a), nullable(b)) from t01;
+----------------------+-----+-----------------------------------------------------------+
| a                    | b   | bit_shift_left(nullable(a), cast(nullable(b) as TINYINT)) |
+----------------------+-----+-----------------------------------------------------------+
| -4388843650233597359 | 127 | -9223372036854775808                                      |
| 306567258363819813   | 3   | 2452538066910558504                                       |
| 306567258363819813   | 127 | -9223372036854775808                                      |
| 3456789123456789123  | 127 | -9223372036854775808                                      |
+----------------------+-----+-----------------------------------------------------------+ 
```
now
```
SELECT a, b, bit_shift_left(nullable(a), nullable(b)) from t01;
+----------------------+-----+-----------------------------------------------------------+
| a                    | b   | bit_shift_left(nullable(a), cast(nullable(b) as TINYINT)) |
+----------------------+-----+-----------------------------------------------------------+
| -4388843650233597359 | 127 | 0                                                         |
| 306567258363819813   | 3   | 2452538066910558504                                       |
| 306567258363819813   | 127 | 0                                                         |
| 3456789123456789123  | 127 | 0                                                         |
+----------------------+-----+-----------------------------------------------------------+
```
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-01-16 21:13:24 +08:00
171251cf42 branch-2.1: [test](mtmv) Modify wrong test method and add some log in debug mode test case #46907 (#47091)
Cherry-picked from #46907

Co-authored-by: seawinde <wusi@selectdb.com>
2025-01-16 17:48:14 +08:00
Pxl
05e0bff137 [Bug](scan) do not ignore non-rf predicate which acting same column with rf #46789 (#47015)
pick from #46789
2025-01-15 16:38:35 +08:00
c6ffa1f4eb [Fix](PreparedStatement) fix incorrect query result with prepared statment (#47012)
cherry-pick from #46994
2025-01-15 14:42:04 +08:00
c60627250d branch-2.1: [Fix](schema change) Fix NPE when rename column on table which has sequence type column #46906 (#46942)
pick https://github.com/apache/doris/pull/46906
2025-01-14 19:25:22 +08:00
a8995b883f [regression-test](case) mv tpcds 100 to nonConccurrent (#46936) 2025-01-14 14:25:01 +08:00
a12f4d7a09 [fix](pipelineX) bp #46493 and add some regresstion tests (#46865)
### What problem does this PR solve?

bp #46493
Add some regresstion tests for #46493 and #46541

Co-authored-by: liulijia <liulijia1029@gmail.com>
2025-01-14 10:11:58 +08:00
6c72bfe082 branch-2.1: [fix](test) fix unstable test after some test migration #46834 (#46852)
Cherry-picked from #46834

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
2025-01-13 18:01:03 +08:00
09411c050b branch-2.1: [Fix](partial update) Forbid partial update when table is in SCHEMA_CHANGE/ROLLUP in FE (#46866)
### What problem does this PR solve?

Related PR: https://github.com/apache/doris/pull/22133

Problem Summary:

https://github.com/apache/doris/pull/22133 forbid doing partial update
during schema change on BE, but this may failed to work when some of
replicas finished alter process on BE and some of them don't. This PR
forbids partial update during schema change on FE to avoid this
situation.

### Release note
2025-01-13 16:00:00 +08:00
eb7fdd5714 [regression-test](fix) fix schema_change_modify_mv_column_type2 case bug (#46878) 2025-01-13 14:02:13 +08:00
7f9ea1a231 [fix](parser) Syntax error in column with agg_state (#46829)
### What problem does this PR solve?

the problem only exists in the old optimizer

```
mysql> CREATE TABLE t (
    ->   `k1` int NULL,
    ->   `k2` agg_state<sum(int null)> GENERIC NOT NULL,
    ->   `k3` agg_state<group_concat(text null)> GENERIC NOT NULL
    -> ) ENGINE=OLAP
    -> AGGREGATE KEY(`k1`)
    -> DISTRIBUTED BY HASH(`k1`) BUCKETS 3
    -> PROPERTIES (
    -> "replication_allocation" = "tag.location.default: 1"
    -> );
ERROR 1105 (HY000): errCode = 2, detailMessage = Syntax error in line 3:
  `k2` agg_state<sum(int null)> GENERIC NOT NULL,
                 ^
Encountered: SUM
Expected: SUM is keyword, maybe `SUM`
```
2025-01-13 11:36:49 +08:00
842196ace3 branch-2.1: [regression-test](fix) schema change case bug #46782 (#46811)
Cherry-picked from #46782

Co-authored-by: shuke <shuke@selectdb.com>
2025-01-12 08:58:14 +08:00
94856b19a2 [fix](hive)fix select count(*) hive full acid tb opt error. (#46732) (#46804)
bp #46732
2025-01-12 08:55:10 +08:00
40f3e9bb77 branch-2.1: [regression-test](fix) fix 'desc table' to follower assertion failure #46788 (#46819)
Cherry-picked from #46788

Co-authored-by: shuke <shuke@selectdb.com>
2025-01-11 22:12:20 +08:00
f618578e9b [regression-test](fix) fix nonConcurrent case bug (#46799) 2025-01-10 22:35:59 +08:00
42808a02dc [Fix](func) fix group_array_intersect result error in all arr contain null (#46590) 2025-01-10 20:14:23 +08:00
d857deb50f branch-2.1: [fix](case) add nonConcurrent to out file path #46679 (#46792)
Cherry-picked from #46679

Co-authored-by: Dongyang Li <lidongyang@selectdb.com>
2025-01-10 19:01:41 +08:00
04b840713b branch-2.1: [fix](jdbc catalog) Use factory methods to return mapped types instead (#46698)
cherry-pick #46623
2025-01-09 18:12:03 +08:00
4f41bad283 branch-2.1: [fix](Nereids) fix cast string to date #46065 (#46526)
pick: #46065

Related PR: #35637

Problem Summary:
When cast("201-01-01" as datetimev2(0)), The result is "2020-01-01" but
it is wrong. It should be result in "0201-01-01".
201 would be regarded as 20xy-0z as related pr show, it was a bug. But
actually it should not have this trasformation and result in
2025-01-09 17:03:36 +08:00
52fd9349f0 [regression-test](case) Reduce test_load_to_single_tablet.groovy data size to prevent failure (#46655) 2025-01-09 08:18:18 +08:00
ca5a4f6d35 [Fix](Variant) variant should not implicit be short key column when create mv (#46539)
cherry-pick from #46444
2025-01-09 08:15:22 +08:00
0ecb84c638 branch-2.1: [Fix](partial update) Fix incorrect result when partial update include delete sign columns #46194 (#46543)
pick https://github.com/apache/doris/pull/46194
2025-01-09 08:14:46 +08:00
4d0037a928 branch-2.1: [fix](ES catalog)Fix query long value exception with doc_value #46554 (#46581)
Cherry-picked from #46554

Co-authored-by: qiye <luen@selectdb.com>
2025-01-08 15:26:58 +08:00
e90480037b [fix](index compaction)Fix construct index compaction columns core when reader close error (#46508)
fix #46507
2025-01-08 10:39:37 +08:00
Pxl
2f98a6216e [Bug](join) fix columnstr64's offset overflow on serialize_value_into_arena #46461 (#46462)
pick from #46461
2025-01-07 19:59:35 +08:00
aa47a35384 [fix](mem) heap-buffer-overflow for function convert_to (#46405) (#46502)
pick #46405 to branch-2.1
2025-01-07 13:46:32 +08:00
8f4b76a5f3 [fix](regression-test) adjust hudi cases (#46473) (#46488)
cherry-pick #46473

Co-authored-by: Socrates <suyiteng@selectdb.com>
2025-01-07 12:11:34 +08:00
12350ca867 branch-2.1: [Improve](mtmv) skip the generation of invalid task for refresh mtmv #46280 (#46392)
Cherry-picked from #46280

Co-authored-by: shee <13843187+qzsee@users.noreply.github.com>
Co-authored-by: garenshi <garenshi@tencent.com>
2025-01-05 08:59:11 +08:00
012848060b branch-2.1: [fix](test) Fix mysql catalog check db test #46263 (#46343)
Cherry-picked from #46263

Co-authored-by: zy-kkk <zhongyk10@gmail.com>
2025-01-04 10:45:26 +08:00
db224ba15f [fix](variant) fix schema change for variant from not null to null (#46403)
cherry-pick from #46279
2025-01-04 09:00:43 +08:00
79955e3d94 branch-2.1: [fix](regression-test) flush out file to fix hudi p2 #46258 (#46317)
Cherry-picked from #46258

Co-authored-by: Socrates <suyiteng@selectdb.com>
2025-01-03 09:13:48 +08:00
7206ca39ec [fix](DECIMAL) error DECIMAL cat to BOOLEAN (#44326) (#46275)
In the past, there were issues with converting `double` and `decimal` to
`boolean`.
For example, a `double` value like 0.13 would first be cast to `uint8`,
resulting in 0.
Then, it would be converted to `bool`, yielding 0 (incorrect, as the
expected result is 1).

Similarly, `decimal` values were directly cast to `uint8`, leading to
non-0/1 values for `bool`.
This issue arises because Doris internally uses `uint8` to represent
`boolean`.

before
```
mysql> select cast(40.123 as BOOLEAN);
+-------------------------+
| cast(40.123 as BOOLEAN) |
+-------------------------+
|                      40 |
+-------------------------+
```

now
```
mysql> select cast(40.123 as BOOLEAN);
+-------------------------+
| cast(40.123 as BOOLEAN) |
+-------------------------+
|                       1 |
+-------------------------+
```

### What problem does this PR solve?

pick #44326
Related PR: #[44326](https://github.com/mrhhsg/doris/tree/pick_44326)

Problem Summary:
2025-01-02 18:42:04 +08:00
7e615afbce branch-2.1: [fix](regression-test) fix test_outfile_csv_with_names_and_types case #46214 (#46272)
Cherry-picked from #46214

Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
2025-01-02 15:06:39 +08:00
b5beaf4eb1 branch-2.1: [fix](mtmv)The refresh method for MTMV is commit. If the status is PAUSED, no more tasks should be generated #46020 (#46265)
Cherry-picked from #46020

Co-authored-by: zhangdong <zhangdong@selectdb.com>
2025-01-02 14:03:28 +08:00
a6fc852ce7 branch-2.1: [opt](catalog) use table in db object return get db #46211 (#46229)
Cherry-picked from #46211

Co-authored-by: zy-kkk <zhongyk10@gmail.com>
Co-authored-by: morningman <yunyou@selectdb.com>
2025-01-01 08:00:49 +08:00
Pxl
43c646363e [Bug](runtime-filter) support ip rf and use exception to replace dche… (#41531)
…ck when PrimitiveType to PColumnType (#39985)

use exception to replace dcheck when PrimitiveType to PColumnType
```cpp
*** SIGABRT unknown detail explain (@0x11d3f) received by PID 73023 (TID 74292 OR 0x7fd758225640) from PID 73023; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_master/doris/be/src/common/signal_handler.h:421
 1# 0x00007FDDBE6B9520 in /lib/x86_64-linux-gnu/libc.so.6
 2# pthread_kill at ./nptl/pthread_kill.c:89
 3# raise at ../sysdeps/posix/raise.c:27
 4# abort at ./stdlib/abort.c:81
 5# 0x000056123F81A94D in /root/output/be/lib/doris_be
 6# 0x000056123F80CF8A in /root/output/be/lib/doris_be
 7# google::LogMessage::SendToLog() in /root/output/be/lib/doris_be
 8# google::LogMessage::Flush() in /root/output/be/lib/doris_be
 9# google::LogMessageFatal::~LogMessageFatal() in /root/output/be/lib/doris_be
10# doris::to_proto(doris::PrimitiveType) at /home/zcp/repo_center/doris_master/doris/be/src/exprs/runtime_filter.cpp:114
11# doris::IRuntimeFilter::push_to_remote(doris::TNetworkAddress const*) at /home/zcp/repo_center/doris_master/doris/be/src/exprs/runtime_filter.cpp:1143
12# doris::IRuntimeFilter::publish(bool)::$_0::operator()(doris::IRuntimeFilter*) const at /home/zcp/repo_center/doris_master/doris/be/src/exprs/runtime_filter.cpp:959
13# doris::IRuntimeFilter::publish(bool)::$_2::operator()() const at /home/zcp/repo_center/doris_master/doris/be/src/exprs/runtime_filter.cpp:983
14# doris::IRuntimeFilter::publish(bool) at /home/zcp/repo_center/doris_master/doris/be/src/exprs/runtime_filter.cpp:997
```

## Proposed changes
pick from #39985
2024-12-30 20:56:11 +08:00
7040abfb04 [fix](correctness) Fix operator initialization (#45728) (#46150)
Now we plan for local exchange depends on operator initialization. This
PR fixed a wrong order between those two steps.

pick #45728
2024-12-30 17:01:33 +08:00
6c9167d9d9 branch-2.1: [test](doc) add job-scheduler example in doris's doc to regression test #42904 (#45968)
Cherry-picked from #42904

Co-authored-by: yagagagaga <zhangminke@selectdb.com>
2024-12-30 10:44:58 +08:00
5425e4686b branch-2.1: [fix](auth) fix use database stmt access unauthorized catalog #45720 (#45978)
Cherry-picked from #45720

Co-authored-by: Petrichor <xiaowenli@selectdb.com>
2024-12-30 10:43:28 +08:00
1d742b5f7d [Cherry-pick](branch-2.1) Pick "[Enhancement](compaction) Do not set failure time when cumulative compaction dealing with delete rowset (#43466)" (#46117)
Before this PR, in cases where there is an alternating distribution of
data rowset -> delete rowset -> data rowset -> delete rowset, cumulative
compaction would only move the cumulative point forward to allow base
compaction to handle the delete rowset. Cumulative compaction itself
would not process the data and would return be marked as failure. This
would cause the compaction submission task process to pause for 5
seconds, impacting efficiency.

This PR modifies the return status to OK for such cases, which improves
the efficiency of the compaction submission task.
2024-12-30 10:18:57 +08:00
a380f5d222 [enchement](utf8)import enable_text_validate_utf8 session var (#45537) (#46070)
bp #45537
2024-12-28 10:05:03 +08:00
4746e9e3a2 [opt](inverted index)Optimize code to get rid of heap use after free (#45745) (#46075)
bp #45745
2024-12-27 16:46:58 +08:00
dd916c0626 [regression-test](fix) fix regression case bugs (#46055)
### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
schema change case bugs when cherry pick
2024-12-27 14:57:25 +08:00
02519c37ea branch-2.1: [test](mysql catalog) fix mysql jdbc catalog test case (#45988) 2024-12-26 15:06:24 +08:00
bceebfcf38 branch-2.1: [test](jdbc catalog) add some jdbc query tvf test (#45991)
cherry-pick #45938
2024-12-26 15:05:50 +08:00
1396f5a950 [2.1][improvement](jdbc catalog) Optimize JdbcCatalog case mapping stability (#43256)
cherry-pick #41510
2024-12-26 14:57:56 +08:00