6dddd4c499
[function](cast)Make string casting to integers more like MySQL's beh… ( #41541 )
...
…avior (#38847 )
https://github.com/apache/doris/pull/38847
## Proposed changes
There are two issues here. First, the results of casting are
inconsistent between FE and BE .
```
FE
mysql [(none)]>select cast('3.000' as int);
+----------------------+
| cast('3.000' as INT) |
+----------------------+
| 3 |
+----------------------+
mysql [(none)]>set debug_skip_fold_constant = true;
BE
mysql [(none)]>select cast('3.000' as int);
+----------------------+
| cast('3.000' as INT) |
+----------------------+
| NULL |
+----------------------+
```
The second issue is that casting on BE converts '3.0' to null. Here, the
casting logic for FE and BE has been unified
<!--Describe your changes.-->
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
---------
Co-authored-by: Xinyi Zou <zouxinyi02@gmail.com >
2024-10-11 09:32:00 +08:00
4c9ebbb3b9
[fix](cloud) cloud group commit should skip repaly wal if label is already used and the txn state is committed or visible ( #41262 ) ( #41461 )
...
pick https://github.com/apache/doris/pull/41262
2024-10-10 22:27:04 +08:00
0fb42d3a48
[Enhancement](tvf)catalog tvf implements user permission checks and hides sensitive information ( #41497 ) ( #41604 )
...
bp #41497
before #21790
## Proposed changes
This PR unifies the duplicate parts of `catalog tvf` and `show
catalogs`, adds permission check when querying `catalog tvf`, and hides
sensitive information.
2024-10-10 17:55:40 +08:00
1db0aef9b7
[feature](array_agg) support array_agg with param is array/map/struct… ( #41651 )
...
… (#40697 )
this pr we support array_agg function support param with array map
struct type
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
2024-10-10 17:54:54 +08:00
070bd6cf99
[fix](jdbc) fix Unknown command(27) ( #41621 ) ( #41637 )
...
pick https://github.com/apache/doris/pull/41621
2024-10-10 17:46:45 +08:00
33fad04341
[opt](Nereids) use 1 instead narrowest column when do column pruning ( #41548 ) ( #41627 )
...
pick from master #41548
2024-10-10 14:02:23 +08:00
15bdfa446c
[case](mtmv)fix alter job case ( #41326 ) ( #41500 )
...
pick: https://github.com/apache/doris/pull/41326
2024-10-10 00:11:50 +08:00
a45dc8796a
[fix](Nereids) simplify decimal comparison wrong when cast to smaller scale ( #41151 ) ( #41618 )
...
pick from master #41151
2024-10-09 23:03:01 +08:00
649cefd70f
[opt](Nereids) forbid distribute under project and filter ( #39812 ) ( #41622 )
...
pick from master #39812
2024-10-09 23:02:06 +08:00
5a6517ba45
[test](inverted index) fix test case for no need read data #41564 ( #41583 )
...
cherry pick from #41564
2024-10-09 14:03:02 +08:00
b6c0603e0d
[fix](mtmv)Fix show mtmv time should wrapped in double quotation mark… ( #41501 )
...
pick: https://github.com/apache/doris/pull/41419
2024-10-09 12:19:28 +08:00
308700f0ca
[fix](test) fix unstable test_export_external_table cases ( #41523 ) ( #41570 )
...
bp #41523
2024-10-09 11:53:22 +08:00
afb477c66d
[Fix](inverted index) Fix wrong need read data opt when enable_common_expr_pushdown is disabled #40689 ( #41562 )
...
cherry pick from #40689
2024-10-08 22:12:10 +08:00
4f81fc474c
[bugfix](paimon)Get the file format by file name ( #41020 ) ( #41487 )
...
bp #41020
2024-09-30 15:46:13 +08:00
03ef90b96e
[regression-test](feature) support skip non-concurrent ( #41493 )
...
…1021)
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
2024-09-30 12:47:18 +08:00
b5e94b65c0
[fix](mtmv) Generate mtmv cache should use ADMIN user, and rewritten plan should not check privilege ( #40374 ) ( #41450 )
...
## Proposed changes
pr: https://github.com/apache/doris/pull/40374
commitId: f3d92e48
2024-09-30 09:59:54 +08:00
d5dccac270
[enhancement](test) Throw exception explicitly in forward query test ( #40742 ) ( #41470 )
...
## Proposed changes
pick: #40742
2024-09-30 09:54:24 +08:00
70630f244e
[fix](regression-test) Add varchar column as distribution key ( #40791 ) ( #41471 )
...
pick: #40791
2024-09-30 09:53:18 +08:00
b7db357847
[test](inverted index) refine test_ignore_above case, add compound query sql #40355 ( #41445 )
...
cherry pick from #40355
2024-09-30 07:04:08 +08:00
9c6122a004
[test](inverted index) fix case for index_compaction #40313 ( #41446 )
...
cherry pick from #40313
2024-09-29 14:08:22 +08:00
d8f3e396ca
[test](inverted index) set enable_common_expr_pushdown true for variant_github_events_p2 #40957 ( #41443 )
...
cherry pick from #40957
2024-09-29 11:31:29 +08:00
d659750fd9
[pick](Serde-2.1) fix variant serde may lost num_rows when subcolumns empty ( #41438 )
...
serialization object with empty subcolumns may lost num_rows, so need to
record num_rows and set back num_rows in serdes
backport #38413
2024-09-29 09:45:37 +08:00
727f0374be
[Refactor](inverted index) refactor inverted index compound predicates evaluate logic #38908 ( #41385 )
...
cherry pick from #38908
2024-09-29 09:19:17 +08:00
05bddfcdd9
[fix](test)resolve table name conflict in test case ( #41373 ) ( #41423 )
...
## Proposed changes
pick from master https://github.com/apache/doris/pull/41373
<!--Describe your changes.-->
2024-09-28 08:36:14 +08:00
705f9c22f0
[2.1][fix](test) fix test_jdbc_query_mysql sql87 ( #41433 )
...
pick (#41158 )
2024-09-27 23:46:17 +08:00
0b4552f74b
[cherry-pick](branch-2.1) pick hive text write from master ( #40537 )
...
## Proposed changes
pick prs:
https://github.com/apache/doris/pull/38549
https://github.com/apache/doris/pull/40183
https://github.com/apache/doris/pull/40315
---------
Co-authored-by: Calvin Kirs <kirs@apache.org >
2024-09-27 20:57:07 +08:00
82228358b9
[Fix](nereids) fix create view with nullable column ( #41234 ) ( #41393 )
...
cherry-pick from master #41234
2024-09-27 19:13:54 +08:00
0c51ee26ea
[fix](function) add time type in conditional-functions ( #41270 ) ( #41379 )
...
## Proposed changes
https://github.com/apache/doris/pull/41270
<!--Describe your changes.-->
2024-09-27 17:19:54 +08:00
1baf0dbc59
[improvement](statistics)Reduce partition column sample BE memory consumption. ( #41203 ) ( #41387 )
...
backport: https://github.com/apache/doris/pull/41203
2024-09-27 13:46:24 +08:00
eb13cd4154
[branch-2.1] Picks "[Fix](partial update) Fix __DORIS_SEQUENCE_COL__ is not set for newly inserted rows in partial update #40272 " ( #40964 )
...
picks https://github.com/apache/doris/pull/40272
2024-09-26 22:54:27 +08:00
bf3d4240be
[fix](window_func) fix bug of agg function used in window function and add many test cases ( #40678 ) ( #41328 )
...
## Proposed changes
Issue Number: close #xxx
BP #40678
2024-09-26 22:50:34 +08:00
e66a6d855d
[case](iceberg)open iceberg write insert && wrrite partition case. ( #41242 )
2024-09-26 22:41:38 +08:00
4ee9547de6
[fix](regression test)Fix unstable case es_catalog_http_open_api ( #41274 ) ( #41332 )
2024-09-26 20:49:35 +08:00
f422b82e24
[Fix](Nereids) fix append_trailing_char_if_absent function return null ( #41157 )
...
cherry-pick： https://github.com/apache/doris/pull/40820
example: select append_trailing_char_if_absent('it','a') would return
null in original design, it can not return null when folding constant on
fe any time
2024-09-26 16:18:12 +08:00
4deda2fce7
[improvement](nereids) Simplify ScanNode projection handling by removing redundant conditions ( #40801 ) ( #41315 )
...
pick from master #40801
This PR simplifies the handling of `ScanNode` projection logic.
Previously, the code included multiple conditional checks to determine
whether a `projectionTuple` should be generated. These conditions have
been removed, and now `projectionTuple `is always generated for
`ScanNode`, ensuring a consistent projection setup. Additionally,
redundant handling of `SlotId` and `SlotRef` has been eliminated, making
the code cleaner and easier to maintain. The behavior for `OlapScanNode`
remains unchanged.
2024-09-26 10:35:01 +08:00
a11fd62043
[fix](window function) Fix illegal frame range ( #41147 ) ( #41305 )
...
pick #41147
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int,
siginfo_t*, void*) at
/home/zcp/repo_center/doris_master/doris/be/src/common/signal_handler.h:421
1# 0x00007F591D573520 in /lib/x86_64-linux-gnu/libc.so.6
2# pthread_kill at ./nptl/pthread_kill.c:89
3# raise at ../sysdeps/posix/raise.c:27
4# abort at ./stdlib/abort.c:81
5# _nl_load_domain at ./intl/loadmsgcat.c:1177
6# 0x00007F591D56AE96 in /lib/x86_64-linux-gnu/libc.so.6
7# doris::vectorized::PODArray<unsigned char, 4096ul, Allocator<false,
false, false, DefaultMemoryAllocator>, 16ul, 15ul>::operator[](long)
const at
/home/zcp/repo_center/doris_master/doris/be/src/vec/common/pod_array.h:365
8# doris::vectorized::ColumnNullable::is_null_at(unsigned long) const at
/home/zcp/repo_center/doris_master/doris/be/src/vec/columns/column_nullable.h:158
9#
doris::vectorized::ReaderFirstAndLastData<doris::vectorized::ColumnVector<double>,
true, true, false>::insert_result_into(doris::vectorized::IColumn&)
const at
/home/zcp/repo_center/doris_master/doris/be/src/vec/aggregate_functions/aggregate_function_reader_first_last.h:125
10# doris::pipeline::AnalyticLocalState::_insert_result_info(long) in
/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be
11# std::_Function_handler<void (long), std::_Bind_result<void, void
(doris::pipeline::AnalyticLocalState::*(doris::pipeline::AnalyticLocalState*,
std::_Placeholder<1>))(long)> >::_M_invoke(std::_Any_data const&,
long&&) at
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
12# std::function<void (long)>::operator()(long) const at
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560
13# doris::pipeline::AnalyticLocalState::_get_next_for_rows(unsigned
long) in /mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be 14#
std::enable_if<is_invocable_r_v<doris::Status, doris::Status
(doris::pipeline::AnalyticLocalState::*&)(unsigned long),
doris::pipeline::AnalyticLocalState*&, unsigned long>,
doris::Status>::type std::__invoke_r<doris::Status, doris::Status
(doris::pipeline::AnalyticLocalState::*&)(unsigned long),
doris::pipeline::AnalyticLocalState*&, unsigned long>(doris::Status
(doris::pipeline::AnalyticLocalState::*&)(unsigned long),
doris::pipeline::AnalyticLocalState*&, unsigned long&&) at
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:114
15# doris::Status std::_Bind_result<doris::Status, doris::Status
(doris::pipeline::AnalyticLocalState::*(doris::pipeline::AnalyticLocalState*,
std::_Placeholder<1>))(unsigned long)>::__call<doris::Status, unsigned
long&&, 0ul, 1ul>(std::tuple<unsigned long&&>&&, std::_Index_tuple<0ul,
1ul>) at
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:570
16# std::_Function_handler<doris::Status (unsigned long),
std::_Bind_result<doris::Status, doris::Status
(doris::pipeline::AnalyticLocalState::*(doris::pipeline::AnalyticLocalState*,
std::_Placeholder<1>))(unsigned long)> >::_M_invoke(std::_Any_data
const&, unsigned long&&) at
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
17# std::function<doris::Status (unsigned long)>::operator()(unsigned
long) const at
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560
18#
doris::pipeline::AnalyticSourceOperatorX::get_block(doris::RuntimeState*,
doris::vectorized::Block*, bool*) in
/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be
19#
doris::pipeline::OperatorXBase::get_block_after_projects(doris::RuntimeState*,
doris::vectorized::Block*, bool*) at
/home/zcp/repo_center/doris_master/doris/be/src/pipeline/exec/operator.cpp:322
20# doris::pipeline::PipelineTask::execute(bool*) in
/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be
21# doris::pipeline::TaskScheduler::_do_work(unsigned long) at
/home/zcp/repo_center/doris_master/doris/be/src/pipeline/task_scheduler.cpp:138
22# doris::ThreadPool::dispatch_thread() in
/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be
23# doris::Thread::supervise_thread(void*) at
/home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:499 24#
start_thread at ./nptl/pthread_create.c:442
25# 0x00007F591D657850 at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:83
2024-09-26 09:55:33 +08:00
ba491af9f1
[branch-2.1] do not fallbacl to origin planner for call stmt ( #41263 )
...
`CALL xxx` statement is not supported by old planner.
And if fallback, the error message will be confused.
2024-09-25 14:30:56 +08:00
5bf210ffa4
[revert](storage) storage medium of partition should not inherit from… ( #41212 )
...
… table (#41192 )
pick: https://github.com/apache/doris/pull/41192
2024-09-25 14:15:24 +08:00
5b3b2cec80
[feat](metatable) support table$partitions for hive table ( #40774 ) ( #41230 )
...
bp #40774
and pick part of #34552 , add `isPartitionedTable()` interface in `TableIf`
2024-09-25 09:52:07 +08:00
fc0b4b9ced
[fix](explain) fix NPE when explain verbose with partition batch mode ( #40969 ) ( #41231 )
...
bp #40969
2024-09-24 21:10:13 +08:00
8bb57bcc3e
[fix] (inverted index) fix the error in the query result when using count on index ( #41200 )
...
## Proposed changes
Introduced by #39473
<!--Describe your changes.-->
2024-09-24 19:47:18 +08:00
3ad9dce067
[case](mtmv) Failed to fix the case of concurrent insert overwrite ex… ( #41163 )
...
…ecution (#41125 )
pick: https://github.com/apache/doris/pull/41125
2024-09-24 16:04:30 +08:00
336a658f9a
[Fix](test)Fixing Kerberos Principal Not Mapping Correctly to a Local User ( #40905 ) ( #41128 )
...
## Proposed changes
error log
http://43.132.222.7:8111/buildConfiguration/Doris_External_Regression/529960?buildTab=tests&status=muted&focusLine=NaN&suite=&package=external_table_p0.kerberos&class=test_single_hive_kerberos&expandedTest=build%3A%28id%3A529960%29%2Cid%3A2000000060
## Changes
- If there are multiple realms, it's best to configure
hadoop.security.auth_to_local to correctly map Kerberos Principals to
local users.
- Thrift exceptions should use those from hive-shade, but to avoid
future changes in hive-shade, I omitted part of the package name prefix.
(cherry picked from commit 902cf1e913f1b01dccb5978e9c9d521a2e8c9350)
#40905
2024-09-24 16:02:51 +08:00
c81182aed8
[fix](restore) Fix clean restore with view #40620 ( #41186 )
...
cherry pick from #40620
2024-09-24 14:10:27 +08:00
ff6f17c22c
[fix](external-p2) ignore external p2 cases( #41148 ) ( #41179 )
...
bp #41148
2024-09-24 09:58:50 +08:00
2b427c316a
[feature](functions) impl scalar functions normal_cdf,to_iso8601,from_iso8601_date ( #40695 ) ( #41049 )
...
bp #40695
2024-09-24 09:52:39 +08:00
8494496dc9
[fix](p2) fix assertion in test_broker_load_p2 ( #41116 ) ( #41117 )
...
cherry-pick #41116
2024-09-24 09:46:07 +08:00
0d38a9a36d
[feature](restore) support atomic restore ( #41107 )
...
Cherry-pick #40353 , #40734 , #40817 , #40876 , #40921 , #41017 , #41083
2024-09-24 09:41:41 +08:00
48e60f3ff3
[Fix](inverted index) fix wrong opt for count_on_index #41127 ( #41154 )
...
cherry pick from #41127
2024-09-23 22:45:52 +08:00
5bcea1983d
[opt](nereids) enable runtime filter use cte as target #40815 (2.1) ( #41090 )
...
## Proposed changes
pick #40815
Issue Number: close #xxx
<!--Describe your changes.-->
2024-09-23 22:34:03 +08:00
