Commit Graph

3564 Commits

Author SHA1 Message Date
04b840713b branch-2.1: [fix](jdbc catalog) Use factory methods to return mapped types instead (#46698)
cherry-pick #46623
2025-01-09 18:12:03 +08:00
4f41bad283 branch-2.1: [fix](Nereids) fix cast string to date #46065 (#46526)
pick: #46065

Related PR: #35637

Problem Summary:
When cast("201-01-01" as datetimev2(0)), The result is "2020-01-01" but
it is wrong. It should be result in "0201-01-01".
201 would be regarded as 20xy-0z as related pr show, it was a bug. But
actually it should not have this trasformation and result in
2025-01-09 17:03:36 +08:00
52fd9349f0 [regression-test](case) Reduce test_load_to_single_tablet.groovy data size to prevent failure (#46655) 2025-01-09 08:18:18 +08:00
ca5a4f6d35 [Fix](Variant) variant should not implicit be short key column when create mv (#46539)
cherry-pick from #46444
2025-01-09 08:15:22 +08:00
0ecb84c638 branch-2.1: [Fix](partial update) Fix incorrect result when partial update include delete sign columns #46194 (#46543)
pick https://github.com/apache/doris/pull/46194
2025-01-09 08:14:46 +08:00
4d0037a928 branch-2.1: [fix](ES catalog)Fix query long value exception with doc_value #46554 (#46581)
Cherry-picked from #46554

Co-authored-by: qiye <luen@selectdb.com>
2025-01-08 15:26:58 +08:00
e90480037b [fix](index compaction)Fix construct index compaction columns core when reader close error (#46508)
fix #46507
2025-01-08 10:39:37 +08:00
Pxl
2f98a6216e [Bug](join) fix columnstr64's offset overflow on serialize_value_into_arena #46461 (#46462)
pick from #46461
2025-01-07 19:59:35 +08:00
aa47a35384 [fix](mem) heap-buffer-overflow for function convert_to (#46405) (#46502)
pick #46405 to branch-2.1
2025-01-07 13:46:32 +08:00
8f4b76a5f3 [fix](regression-test) adjust hudi cases (#46473) (#46488)
cherry-pick #46473

Co-authored-by: Socrates <suyiteng@selectdb.com>
2025-01-07 12:11:34 +08:00
12350ca867 branch-2.1: [Improve](mtmv) skip the generation of invalid task for refresh mtmv #46280 (#46392)
Cherry-picked from #46280

Co-authored-by: shee <13843187+qzsee@users.noreply.github.com>
Co-authored-by: garenshi <garenshi@tencent.com>
2025-01-05 08:59:11 +08:00
012848060b branch-2.1: [fix](test) Fix mysql catalog check db test #46263 (#46343)
Cherry-picked from #46263

Co-authored-by: zy-kkk <zhongyk10@gmail.com>
2025-01-04 10:45:26 +08:00
db224ba15f [fix](variant) fix schema change for variant from not null to null (#46403)
cherry-pick from #46279
2025-01-04 09:00:43 +08:00
79955e3d94 branch-2.1: [fix](regression-test) flush out file to fix hudi p2 #46258 (#46317)
Cherry-picked from #46258

Co-authored-by: Socrates <suyiteng@selectdb.com>
2025-01-03 09:13:48 +08:00
7206ca39ec [fix](DECIMAL) error DECIMAL cat to BOOLEAN (#44326) (#46275)
In the past, there were issues with converting `double` and `decimal` to
`boolean`.
For example, a `double` value like 0.13 would first be cast to `uint8`,
resulting in 0.
Then, it would be converted to `bool`, yielding 0 (incorrect, as the
expected result is 1).

Similarly, `decimal` values were directly cast to `uint8`, leading to
non-0/1 values for `bool`.
This issue arises because Doris internally uses `uint8` to represent
`boolean`.

before
```
mysql> select cast(40.123 as BOOLEAN);
+-------------------------+
| cast(40.123 as BOOLEAN) |
+-------------------------+
|                      40 |
+-------------------------+
```

now
```
mysql> select cast(40.123 as BOOLEAN);
+-------------------------+
| cast(40.123 as BOOLEAN) |
+-------------------------+
|                       1 |
+-------------------------+
```

### What problem does this PR solve?

pick #44326
Related PR: #[44326](https://github.com/mrhhsg/doris/tree/pick_44326)

Problem Summary:
2025-01-02 18:42:04 +08:00
7e615afbce branch-2.1: [fix](regression-test) fix test_outfile_csv_with_names_and_types case #46214 (#46272)
Cherry-picked from #46214

Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
2025-01-02 15:06:39 +08:00
b5beaf4eb1 branch-2.1: [fix](mtmv)The refresh method for MTMV is commit. If the status is PAUSED, no more tasks should be generated #46020 (#46265)
Cherry-picked from #46020

Co-authored-by: zhangdong <zhangdong@selectdb.com>
2025-01-02 14:03:28 +08:00
a6fc852ce7 branch-2.1: [opt](catalog) use table in db object return get db #46211 (#46229)
Cherry-picked from #46211

Co-authored-by: zy-kkk <zhongyk10@gmail.com>
Co-authored-by: morningman <yunyou@selectdb.com>
2025-01-01 08:00:49 +08:00
Pxl
43c646363e [Bug](runtime-filter) support ip rf and use exception to replace dche… (#41531)
…ck when PrimitiveType to PColumnType (#39985)

use exception to replace dcheck when PrimitiveType to PColumnType
```cpp
*** SIGABRT unknown detail explain (@0x11d3f) received by PID 73023 (TID 74292 OR 0x7fd758225640) from PID 73023; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_master/doris/be/src/common/signal_handler.h:421
 1# 0x00007FDDBE6B9520 in /lib/x86_64-linux-gnu/libc.so.6
 2# pthread_kill at ./nptl/pthread_kill.c:89
 3# raise at ../sysdeps/posix/raise.c:27
 4# abort at ./stdlib/abort.c:81
 5# 0x000056123F81A94D in /root/output/be/lib/doris_be
 6# 0x000056123F80CF8A in /root/output/be/lib/doris_be
 7# google::LogMessage::SendToLog() in /root/output/be/lib/doris_be
 8# google::LogMessage::Flush() in /root/output/be/lib/doris_be
 9# google::LogMessageFatal::~LogMessageFatal() in /root/output/be/lib/doris_be
10# doris::to_proto(doris::PrimitiveType) at /home/zcp/repo_center/doris_master/doris/be/src/exprs/runtime_filter.cpp:114
11# doris::IRuntimeFilter::push_to_remote(doris::TNetworkAddress const*) at /home/zcp/repo_center/doris_master/doris/be/src/exprs/runtime_filter.cpp:1143
12# doris::IRuntimeFilter::publish(bool)::$_0::operator()(doris::IRuntimeFilter*) const at /home/zcp/repo_center/doris_master/doris/be/src/exprs/runtime_filter.cpp:959
13# doris::IRuntimeFilter::publish(bool)::$_2::operator()() const at /home/zcp/repo_center/doris_master/doris/be/src/exprs/runtime_filter.cpp:983
14# doris::IRuntimeFilter::publish(bool) at /home/zcp/repo_center/doris_master/doris/be/src/exprs/runtime_filter.cpp:997
```

## Proposed changes
pick from #39985
2024-12-30 20:56:11 +08:00
7040abfb04 [fix](correctness) Fix operator initialization (#45728) (#46150)
Now we plan for local exchange depends on operator initialization. This
PR fixed a wrong order between those two steps.

pick #45728
2024-12-30 17:01:33 +08:00
6c9167d9d9 branch-2.1: [test](doc) add job-scheduler example in doris's doc to regression test #42904 (#45968)
Cherry-picked from #42904

Co-authored-by: yagagagaga <zhangminke@selectdb.com>
2024-12-30 10:44:58 +08:00
5425e4686b branch-2.1: [fix](auth) fix use database stmt access unauthorized catalog #45720 (#45978)
Cherry-picked from #45720

Co-authored-by: Petrichor <xiaowenli@selectdb.com>
2024-12-30 10:43:28 +08:00
1d742b5f7d [Cherry-pick](branch-2.1) Pick "[Enhancement](compaction) Do not set failure time when cumulative compaction dealing with delete rowset (#43466)" (#46117)
Before this PR, in cases where there is an alternating distribution of
data rowset -> delete rowset -> data rowset -> delete rowset, cumulative
compaction would only move the cumulative point forward to allow base
compaction to handle the delete rowset. Cumulative compaction itself
would not process the data and would return be marked as failure. This
would cause the compaction submission task process to pause for 5
seconds, impacting efficiency.

This PR modifies the return status to OK for such cases, which improves
the efficiency of the compaction submission task.
2024-12-30 10:18:57 +08:00
a380f5d222 [enchement](utf8)import enable_text_validate_utf8 session var (#45537) (#46070)
bp #45537
2024-12-28 10:05:03 +08:00
4746e9e3a2 [opt](inverted index)Optimize code to get rid of heap use after free (#45745) (#46075)
bp #45745
2024-12-27 16:46:58 +08:00
dd916c0626 [regression-test](fix) fix regression case bugs (#46055)
### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
schema change case bugs when cherry pick
2024-12-27 14:57:25 +08:00
02519c37ea branch-2.1: [test](mysql catalog) fix mysql jdbc catalog test case (#45988) 2024-12-26 15:06:24 +08:00
bceebfcf38 branch-2.1: [test](jdbc catalog) add some jdbc query tvf test (#45991)
cherry-pick #45938
2024-12-26 15:05:50 +08:00
1396f5a950 [2.1][improvement](jdbc catalog) Optimize JdbcCatalog case mapping stability (#43256)
cherry-pick #41510
2024-12-26 14:57:56 +08:00
289d621faa [improvement](information_schema)Show view definition in information_schema.views. (#45857) (#45930)
backport: https://github.com/apache/doris/pull/45857
2024-12-26 10:11:13 +08:00
547e88b1ee branch-2.1: [fix](csv reader) fix core dump when parsing csv with enclose #45485 (#45889)
Cherry-picked from #45485

Co-authored-by: hui lai <laihui@selectdb.com>
2024-12-25 12:09:20 +08:00
fdb65197e7 [feat](mtmv)mtmv support paimon partition refresh(#43959,#44911) (#45878)
pick: #44911 #43959

only pick code about paimon, not pick some code about MTMV REFRESH
2024-12-25 12:08:27 +08:00
6c2107788c [fix](nereids) support one phase DeferMaterializeTopN #45693-branch-2.1 (#45845) 2024-12-25 12:00:58 +08:00
f0031d9954 [cherry-pick](branch-21)support posexplode table function (#43221) (#45783)
cherry-pick from master #43221
2024-12-24 21:42:30 +08:00
0a64cea3bf [fix](mtmv) Fix mv is deleted in nested mv causing query err and fix some test (#45744) (#45862)
pr: 
https://github.com/apache/doris/pull/45744

commitId: 067643df
2024-12-24 20:03:24 +08:00
c9d3f1122b [regression-test](framework) disable defining global variable in test… (#45840) 2024-12-24 17:21:27 +08:00
8b35b0e477 branch-2.1: [fix](function) fixed some nested type func's param type which is not suitable and make result wrong #44923 (#45798)
Cherry-picked from #44923
2024-12-24 14:57:33 +08:00
ca9f0ade25 [Fix](ShortCircuit) fix prepared statement with partial arguments prepared (#45371) (#45732)
(#45371)
2024-12-23 10:03:34 +08:00
303557ac70 [fix](hive)fix hive insert only translaction table. (#45753)
### What problem does this PR solve?
bp #44001 , but no hive4 acid table.

Problem Summary:
1. Fixed the issue that when reading insert translaction only tables,
there was no acid check, which caused multiple data reads (i.e., reading
data from the previous base_n).
2. Forbidden to create, insert data, and delete aicd tables.
2024-12-22 21:23:21 +08:00
f733aa5dcf branch-2.1: [fix](fold) fixed an issue with be computing constants #43410 (#45712)
Cherry-picked from #43410
2024-12-20 20:26:27 +08:00
19c0e89da7 [enchement](iceberg)support read iceberg partition evolution table. (#45367) (#45569)
cherry-pick #45367

Co-authored-by: daidai <changyuwei@selectdb.com>
2024-12-20 08:56:51 +08:00
4b7c2eaa7d [branch-2.1](fix) fix incorrect result of hash join with const column (#45630) 2024-12-19 19:14:38 +08:00
f966088fd0 branch-2.1: [fix](nereids) fix ExtractAndNormalizeWindowExpression bug #45553 (#45635)
Cherry-picked from #45553

Co-authored-by: feiniaofeiafei <moailing@selectdb.com>
2024-12-19 19:13:03 +08:00
55e5e69a81 [branch-2.1]Fix test_audit_log_behavior failed (#45636) 2024-12-19 17:43:11 +08:00
d6c629d293 branch-2.1: [fix](tvf) Tvf supports to parse the enclose character in csv files #45407 (#45570)
Cherry-picked from #45407

Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
2024-12-19 16:23:44 +08:00
7957e6fe0f branch-2.1: [Fix] Add column with default value string throw exception #45000 (#45316)
Cherry-picked from #45000

Co-authored-by: Uniqueyou <134280716+wyxxxcat@users.noreply.github.com>
2024-12-19 14:07:43 +08:00
7d32e4f71f branch-2.1: [Fix](ORC) Not push down fixed char type in orc reader #45484 (#45525)
cherry-pick #45484
2024-12-19 14:06:00 +08:00
afeb5863ad branch-2.1: [fix](nereids) fix compare with date like literal #45382 (#45595)
cherry pick from #45382
2024-12-18 22:59:27 +08:00
Pxl
fe0cc289de [Bug](function) fix wrong result on group_concat with distinct+order_… (#45513) 2024-12-18 22:49:18 +08:00
1980e098bc branch-2.1: [Test](binlog) Check correctly for show create database #45472 (#45551)
Cherry-picked from #45472

Co-authored-by: Uniqueyou <wangyixuan@selectdb.com>
2024-12-18 22:43:15 +08:00