Commit Graph

3593 Commits

Author SHA1 Message Date
2f7e0f0b76 Revert "[branch-2.1]downgrade resource tag when there is not queryabl… (#47746)
revert #45387
2025-02-11 16:51:07 +08:00
6ceb010195 branch-2.1: [fix](mv)Fix direct select mv pre agg bug. #47658 (#47700)
Cherry-picked from #47658

Co-authored-by: James <lijibing@selectdb.com>
2025-02-10 21:45:00 +08:00
f4e92d75f5 [branch-2.1] fix select * from variables system table (#34529) (#47660)
pick #34529

Signed-off-by: nextdreamblue <zxw520blue1@163.com>
Co-authored-by: xueweizhang <zxw520blue1@163.com>
2025-02-09 14:14:06 +08:00
803d3a1545 Revert "[fix](date_function) fix str_to_date function return wrong microsecond issue" (#47441) 2025-01-26 12:53:11 +08:00
2ae03d7e02 branch-2.1: [Bug](function) fix wrong result when case when have more than 256 conditions and the… #47179 (#47392)
Cherry-picked from #47179

Co-authored-by: Pxl <xl@selectdb.com>
2025-01-25 10:04:50 +08:00
dd7affb8f9 branch-2.1 [Fix](ShortCircuite) fix point query crash with prepared statement when encounter delete sign (#47227)
cherry-pick from #47178
2025-01-25 08:33:06 +08:00
3b7cc67ec2 branch-2.1: [fix](func) Fix precision loss in ST_GeometryFromWKB coordinate parsing #46661 (#47263)
Cherry-picked from #46661

Co-authored-by: lw112 <131352377+felixwluo@users.noreply.github.com>
2025-01-24 17:34:37 +08:00
8925a390d6 [fix](date_function) fix str_to_date function return wrong microsecond issue (#47252) 2025-01-24 17:32:23 +08:00
3aad9e5f67 [opt](oceanbase docker)Use LTS docker image and print unhealthy docker logs (#46647) (#47349)
### What problem does this PR solve?

Problem Summary:
Oceanbase container sometimes start failed.
<img width="653" alt="image"

src="https://github.com/user-attachments/assets/d95c66cf-7e04-4179-a565-9b9dd8b87128"
/>

We do two things:
1. Print last 100 lines docker logs of unhealthy container for debugging
2. Upgrade Oceanbase docker image to the newest `4.2.1-lts`, since it is
7 months newer than `4.2.1`, more stable
2025-01-24 11:22:02 +08:00
4f9562650d Branch-2.1 [Fix](Variant) fix variant serialize to string (#47121) (#47147)
cherry-pick from #47121
2025-01-18 09:12:39 +08:00
9dffe5992e branch-2.1: [fix](json) fix parsing double in jsonb #46977 (#47065)
Cherry-picked from #46977

Co-authored-by: Sun Chenyang <sunchenyang@selectdb.com>
2025-01-18 00:25:13 +08:00
f2fe07a0b9 [fix](regression) join condition is missing in tpcds query24 (#47158) 2025-01-17 20:31:28 +08:00
adc9e9a161 [fix](function) Fix undefined behavior caused by bit_shift_left exceeding the 64-bit range. (#47080)
### What problem does this PR solve?
It is part of this PR's https://github.com/apache/doris/pull/35769
content.


```
SELECT a, b, bit_shift_left(nullable(a), nullable(b)) from t01;
+----------------------+-----+-----------------------------------------------------------+
| a                    | b   | bit_shift_left(nullable(a), cast(nullable(b) as TINYINT)) |
+----------------------+-----+-----------------------------------------------------------+
| -4388843650233597359 | 127 | -9223372036854775808                                      |
| 306567258363819813   | 3   | 2452538066910558504                                       |
| 306567258363819813   | 127 | -9223372036854775808                                      |
| 3456789123456789123  | 127 | -9223372036854775808                                      |
+----------------------+-----+-----------------------------------------------------------+ 
```
now
```
SELECT a, b, bit_shift_left(nullable(a), nullable(b)) from t01;
+----------------------+-----+-----------------------------------------------------------+
| a                    | b   | bit_shift_left(nullable(a), cast(nullable(b) as TINYINT)) |
+----------------------+-----+-----------------------------------------------------------+
| -4388843650233597359 | 127 | 0                                                         |
| 306567258363819813   | 3   | 2452538066910558504                                       |
| 306567258363819813   | 127 | 0                                                         |
| 3456789123456789123  | 127 | 0                                                         |
+----------------------+-----+-----------------------------------------------------------+
```
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2025-01-16 21:13:24 +08:00
171251cf42 branch-2.1: [test](mtmv) Modify wrong test method and add some log in debug mode test case #46907 (#47091)
Cherry-picked from #46907

Co-authored-by: seawinde <wusi@selectdb.com>
2025-01-16 17:48:14 +08:00
Pxl
05e0bff137 [Bug](scan) do not ignore non-rf predicate which acting same column with rf #46789 (#47015)
pick from #46789
2025-01-15 16:38:35 +08:00
c6ffa1f4eb [Fix](PreparedStatement) fix incorrect query result with prepared statment (#47012)
cherry-pick from #46994
2025-01-15 14:42:04 +08:00
c60627250d branch-2.1: [Fix](schema change) Fix NPE when rename column on table which has sequence type column #46906 (#46942)
pick https://github.com/apache/doris/pull/46906
2025-01-14 19:25:22 +08:00
a8995b883f [regression-test](case) mv tpcds 100 to nonConccurrent (#46936) 2025-01-14 14:25:01 +08:00
a12f4d7a09 [fix](pipelineX) bp #46493 and add some regresstion tests (#46865)
### What problem does this PR solve?

bp #46493
Add some regresstion tests for #46493 and #46541

Co-authored-by: liulijia <liulijia1029@gmail.com>
2025-01-14 10:11:58 +08:00
6c72bfe082 branch-2.1: [fix](test) fix unstable test after some test migration #46834 (#46852)
Cherry-picked from #46834

Co-authored-by: Mingyu Chen (Rayner) <morningman@163.com>
2025-01-13 18:01:03 +08:00
09411c050b branch-2.1: [Fix](partial update) Forbid partial update when table is in SCHEMA_CHANGE/ROLLUP in FE (#46866)
### What problem does this PR solve?

Related PR: https://github.com/apache/doris/pull/22133

Problem Summary:

https://github.com/apache/doris/pull/22133 forbid doing partial update
during schema change on BE, but this may failed to work when some of
replicas finished alter process on BE and some of them don't. This PR
forbids partial update during schema change on FE to avoid this
situation.

### Release note
2025-01-13 16:00:00 +08:00
eb7fdd5714 [regression-test](fix) fix schema_change_modify_mv_column_type2 case bug (#46878) 2025-01-13 14:02:13 +08:00
7f9ea1a231 [fix](parser) Syntax error in column with agg_state (#46829)
### What problem does this PR solve?

the problem only exists in the old optimizer

```
mysql> CREATE TABLE t (
    ->   `k1` int NULL,
    ->   `k2` agg_state<sum(int null)> GENERIC NOT NULL,
    ->   `k3` agg_state<group_concat(text null)> GENERIC NOT NULL
    -> ) ENGINE=OLAP
    -> AGGREGATE KEY(`k1`)
    -> DISTRIBUTED BY HASH(`k1`) BUCKETS 3
    -> PROPERTIES (
    -> "replication_allocation" = "tag.location.default: 1"
    -> );
ERROR 1105 (HY000): errCode = 2, detailMessage = Syntax error in line 3:
  `k2` agg_state<sum(int null)> GENERIC NOT NULL,
                 ^
Encountered: SUM
Expected: SUM is keyword, maybe `SUM`
```
2025-01-13 11:36:49 +08:00
842196ace3 branch-2.1: [regression-test](fix) schema change case bug #46782 (#46811)
Cherry-picked from #46782

Co-authored-by: shuke <shuke@selectdb.com>
2025-01-12 08:58:14 +08:00
94856b19a2 [fix](hive)fix select count(*) hive full acid tb opt error. (#46732) (#46804)
bp #46732
2025-01-12 08:55:10 +08:00
40f3e9bb77 branch-2.1: [regression-test](fix) fix 'desc table' to follower assertion failure #46788 (#46819)
Cherry-picked from #46788

Co-authored-by: shuke <shuke@selectdb.com>
2025-01-11 22:12:20 +08:00
f618578e9b [regression-test](fix) fix nonConcurrent case bug (#46799) 2025-01-10 22:35:59 +08:00
42808a02dc [Fix](func) fix group_array_intersect result error in all arr contain null (#46590) 2025-01-10 20:14:23 +08:00
d857deb50f branch-2.1: [fix](case) add nonConcurrent to out file path #46679 (#46792)
Cherry-picked from #46679

Co-authored-by: Dongyang Li <lidongyang@selectdb.com>
2025-01-10 19:01:41 +08:00
04b840713b branch-2.1: [fix](jdbc catalog) Use factory methods to return mapped types instead (#46698)
cherry-pick #46623
2025-01-09 18:12:03 +08:00
4f41bad283 branch-2.1: [fix](Nereids) fix cast string to date #46065 (#46526)
pick: #46065

Related PR: #35637

Problem Summary:
When cast("201-01-01" as datetimev2(0)), The result is "2020-01-01" but
it is wrong. It should be result in "0201-01-01".
201 would be regarded as 20xy-0z as related pr show, it was a bug. But
actually it should not have this trasformation and result in
2025-01-09 17:03:36 +08:00
52fd9349f0 [regression-test](case) Reduce test_load_to_single_tablet.groovy data size to prevent failure (#46655) 2025-01-09 08:18:18 +08:00
ca5a4f6d35 [Fix](Variant) variant should not implicit be short key column when create mv (#46539)
cherry-pick from #46444
2025-01-09 08:15:22 +08:00
0ecb84c638 branch-2.1: [Fix](partial update) Fix incorrect result when partial update include delete sign columns #46194 (#46543)
pick https://github.com/apache/doris/pull/46194
2025-01-09 08:14:46 +08:00
4d0037a928 branch-2.1: [fix](ES catalog)Fix query long value exception with doc_value #46554 (#46581)
Cherry-picked from #46554

Co-authored-by: qiye <luen@selectdb.com>
2025-01-08 15:26:58 +08:00
e90480037b [fix](index compaction)Fix construct index compaction columns core when reader close error (#46508)
fix #46507
2025-01-08 10:39:37 +08:00
Pxl
2f98a6216e [Bug](join) fix columnstr64's offset overflow on serialize_value_into_arena #46461 (#46462)
pick from #46461
2025-01-07 19:59:35 +08:00
aa47a35384 [fix](mem) heap-buffer-overflow for function convert_to (#46405) (#46502)
pick #46405 to branch-2.1
2025-01-07 13:46:32 +08:00
8f4b76a5f3 [fix](regression-test) adjust hudi cases (#46473) (#46488)
cherry-pick #46473

Co-authored-by: Socrates <suyiteng@selectdb.com>
2025-01-07 12:11:34 +08:00
12350ca867 branch-2.1: [Improve](mtmv) skip the generation of invalid task for refresh mtmv #46280 (#46392)
Cherry-picked from #46280

Co-authored-by: shee <13843187+qzsee@users.noreply.github.com>
Co-authored-by: garenshi <garenshi@tencent.com>
2025-01-05 08:59:11 +08:00
012848060b branch-2.1: [fix](test) Fix mysql catalog check db test #46263 (#46343)
Cherry-picked from #46263

Co-authored-by: zy-kkk <zhongyk10@gmail.com>
2025-01-04 10:45:26 +08:00
db224ba15f [fix](variant) fix schema change for variant from not null to null (#46403)
cherry-pick from #46279
2025-01-04 09:00:43 +08:00
79955e3d94 branch-2.1: [fix](regression-test) flush out file to fix hudi p2 #46258 (#46317)
Cherry-picked from #46258

Co-authored-by: Socrates <suyiteng@selectdb.com>
2025-01-03 09:13:48 +08:00
7206ca39ec [fix](DECIMAL) error DECIMAL cat to BOOLEAN (#44326) (#46275)
In the past, there were issues with converting `double` and `decimal` to
`boolean`.
For example, a `double` value like 0.13 would first be cast to `uint8`,
resulting in 0.
Then, it would be converted to `bool`, yielding 0 (incorrect, as the
expected result is 1).

Similarly, `decimal` values were directly cast to `uint8`, leading to
non-0/1 values for `bool`.
This issue arises because Doris internally uses `uint8` to represent
`boolean`.

before
```
mysql> select cast(40.123 as BOOLEAN);
+-------------------------+
| cast(40.123 as BOOLEAN) |
+-------------------------+
|                      40 |
+-------------------------+
```

now
```
mysql> select cast(40.123 as BOOLEAN);
+-------------------------+
| cast(40.123 as BOOLEAN) |
+-------------------------+
|                       1 |
+-------------------------+
```

### What problem does this PR solve?

pick #44326
Related PR: #[44326](https://github.com/mrhhsg/doris/tree/pick_44326)

Problem Summary:
2025-01-02 18:42:04 +08:00
7e615afbce branch-2.1: [fix](regression-test) fix test_outfile_csv_with_names_and_types case #46214 (#46272)
Cherry-picked from #46214

Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
2025-01-02 15:06:39 +08:00
b5beaf4eb1 branch-2.1: [fix](mtmv)The refresh method for MTMV is commit. If the status is PAUSED, no more tasks should be generated #46020 (#46265)
Cherry-picked from #46020

Co-authored-by: zhangdong <zhangdong@selectdb.com>
2025-01-02 14:03:28 +08:00
a6fc852ce7 branch-2.1: [opt](catalog) use table in db object return get db #46211 (#46229)
Cherry-picked from #46211

Co-authored-by: zy-kkk <zhongyk10@gmail.com>
Co-authored-by: morningman <yunyou@selectdb.com>
2025-01-01 08:00:49 +08:00
Pxl
43c646363e [Bug](runtime-filter) support ip rf and use exception to replace dche… (#41531)
…ck when PrimitiveType to PColumnType (#39985)

use exception to replace dcheck when PrimitiveType to PColumnType
```cpp
*** SIGABRT unknown detail explain (@0x11d3f) received by PID 73023 (TID 74292 OR 0x7fd758225640) from PID 73023; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_master/doris/be/src/common/signal_handler.h:421
 1# 0x00007FDDBE6B9520 in /lib/x86_64-linux-gnu/libc.so.6
 2# pthread_kill at ./nptl/pthread_kill.c:89
 3# raise at ../sysdeps/posix/raise.c:27
 4# abort at ./stdlib/abort.c:81
 5# 0x000056123F81A94D in /root/output/be/lib/doris_be
 6# 0x000056123F80CF8A in /root/output/be/lib/doris_be
 7# google::LogMessage::SendToLog() in /root/output/be/lib/doris_be
 8# google::LogMessage::Flush() in /root/output/be/lib/doris_be
 9# google::LogMessageFatal::~LogMessageFatal() in /root/output/be/lib/doris_be
10# doris::to_proto(doris::PrimitiveType) at /home/zcp/repo_center/doris_master/doris/be/src/exprs/runtime_filter.cpp:114
11# doris::IRuntimeFilter::push_to_remote(doris::TNetworkAddress const*) at /home/zcp/repo_center/doris_master/doris/be/src/exprs/runtime_filter.cpp:1143
12# doris::IRuntimeFilter::publish(bool)::$_0::operator()(doris::IRuntimeFilter*) const at /home/zcp/repo_center/doris_master/doris/be/src/exprs/runtime_filter.cpp:959
13# doris::IRuntimeFilter::publish(bool)::$_2::operator()() const at /home/zcp/repo_center/doris_master/doris/be/src/exprs/runtime_filter.cpp:983
14# doris::IRuntimeFilter::publish(bool) at /home/zcp/repo_center/doris_master/doris/be/src/exprs/runtime_filter.cpp:997
```

## Proposed changes
pick from #39985
2024-12-30 20:56:11 +08:00
7040abfb04 [fix](correctness) Fix operator initialization (#45728) (#46150)
Now we plan for local exchange depends on operator initialization. This
PR fixed a wrong order between those two steps.

pick #45728
2024-12-30 17:01:33 +08:00
6c9167d9d9 branch-2.1: [test](doc) add job-scheduler example in doris's doc to regression test #42904 (#45968)
Cherry-picked from #42904

Co-authored-by: yagagagaga <zhangminke@selectdb.com>
2024-12-30 10:44:58 +08:00