Commit Graph

19125 Commits

Author SHA1 Message Date
8fc70e32bc [fix](planner) fix wrong resut of function ifnull/coalesce caused by … (#36727) 2024-06-25 17:32:04 +08:00
695d58f354 [cherry-pick](scan)scanner could eos early when reached limit (#36535) (#36736)
## Proposed changes
cherry-pick from master #36535
2024-06-25 17:22:43 +08:00
11201feae5 [fix](spill join) fix coredump of debug_string (#36723)
## Proposed changes

Pick #36715

<!--Describe your changes.-->
2024-06-25 16:33:33 +08:00
785a1f49f5 [fix](txn) Fix coordidator be restart not abort txn #35342 (#36437)
cherry pick from #35342
2024-06-25 13:35:01 +08:00
07ce9cf52c [fix](schema change) reduce memory usage in schema change process #30231 #36285 #33073 (#36756)
pick
https://github.com/apache/doris/pull/30231
https://github.com/apache/doris/pull/36285
https://github.com/apache/doris/pull/33073
2024-06-25 12:21:17 +08:00
3652fc31c3 [Pick 2.1] "Fix data loss when node channel been cancelled before close wait (#36662)" (#36744)
## Proposed changes

Pick from https://github.com/apache/doris/pull/36662
2024-06-25 11:36:31 +08:00
6ec9a731e8 [branch-2.1](cherry-pick) partial update should not read old fileds from rows with delete sign (#36210) (#36755)
cherry-pick #36210
2024-06-24 21:13:24 +08:00
67adbdae75 [branch-2.1] Pick "[Fix](JournalEntity) re-add a line of code that is accidentally removed in #19917" (#36427)
## Proposed changes

pick https://github.com/apache/doris/pull/36423
2024-06-24 20:45:02 +08:00
e0088df3b3 [case](udf) support run java udf case on cluster with multiple BEs (#… (#36742)
…36669)

Issue Number: close #xxx

<!--Describe your changes.-->

Co-authored-by: stephen <hello-stephen@qq.com>
2024-06-24 17:32:08 +08:00
e4b6dac0c1 [fix](ubsan) reinterpret_cast fix length types to int8 is not safe (#36725)
## Proposed changes

Fix type check of ubsan. 
```
/root/doris/be/src/vec/exec/format/parquet/fix_length_plain_decoder.h:75:78: runtime error: member call on address 0x5582f35db5c0 which does not point to an object of type 'doris::vectorized::ColumnVector<signed char>'
0x5582f35db5c0: note: object is of type 'doris::vectorized::ColumnVector<int>'
 83 55 00 00  78 c0 b0 5a 82 55 00 00  02 00 00 00 00 00 00 00  10 a0 00 d7 83 55 00 00  10 a0 00 d7
              ^~~~~~~~~~~~~~~~~~~~~~~
              vptr for 'doris::vectorized::ColumnVector<int>'
doris::Status doris::vectorized::FixLengthPlainDecoder::_decode_values<false>(COW<doris::vectorized::IColumn>::mutable_ptr<doris::vectorized::IColumn>&, std::shared_ptr<doris::vectorized::IDataType const>&, doris::vectorized::ColumnSelectVector&, bool) at fix_length_plain_decoder.h:75:78
```
2024-06-24 14:03:41 +08:00
Pxl
c6205783fa [Bug](function) fix wrong output_char_size on hll_to_base64 (#36572)
## Proposed changes
pick from #36529
2024-06-24 13:19:28 +08:00
aeec08639d [branch-2.1](tag) 2.1.4-rc03 (#36706)
change version to 2.1.4-rc03
2024-06-23 00:26:13 +08:00
02fad48870 [Fix](upgrade) Fix fields not handled correctly during upgrade and downgrade (#36691)
master version is #36690
2024-06-22 14:23:04 +08:00
17cf34b244 [Fix](multi-catalog) Fix core in orc and parquet reader sometimes after low mem exception. (#36575)
## Proposed changes

Backport #36574.
2024-06-22 11:28:21 +08:00
90a4dd09f3 [Fix](func) CoreDump and Result Error in percentile function (#36647)
cherry pick #36643
2024-06-21 23:42:45 +08:00
445d42a57d [fix](topn-opt) remove redundant check for fetch phase (#36676)
#36629
Issue Number: close #xxx

<!--Describe your changes.-->
2024-06-21 22:28:38 +08:00
c8e4c404fa [Fix]check if fe set thrift field current_connect_fe (#36681)
bp #36678
2024-06-21 22:15:25 +08:00
c939781411 [Pick 2.1](inverted index) fix wrong no need read data when need_remaining_after_evaluate (#36684)
When using an equal predicate on a column that applies an inverted index
with a parser, it requires remaining_after_evaluate. In this situation,
we cannot optimize the column without reading the data.

## Proposed changes

From (#36637)
2024-06-21 22:01:39 +08:00
0cff539810 [feature](function) support new function replace_empty (#36283) (#36656)
#36283
2024-06-21 16:46:22 +08:00
c8f2a3f952 [fix](eq_for_null) fix incorrect logic in function eq_for_null #36004 (#36124)
cherry pick from #36004
cherry pick from #36164
2024-06-21 14:31:21 +08:00
8105dc7de8 [Pick 2.1](inverted index) fix wrong opt for pk no need read data (#36634)
## Proposed changes
 
Pick from #36618
2024-06-21 00:57:23 +08:00
58cc1dca7f [improve](fe) Support to config max msg/frame size of the thrift server (#36594)
Cherry-pick #35845
2024-06-21 00:15:15 +08:00
3febac1d91 [fix](connection) kill connection when meeting Write mysql packet failed error #36559 (#36616)
bp #36559
2024-06-20 22:27:01 +08:00
c28c243c98 [Fix](Variant) forbit create variant as key #36555 (#36578) 2024-06-20 20:33:48 +08:00
a79b56ac23 [chore](be) Support config max message size for be thrift server (#36595)
Cherry-pick #36467
2024-06-20 20:15:43 +08:00
b3dcfae864 [chore](be) Improve ingesting binlog error checking (#36596)
Cherry-pick #36487
2024-06-20 20:15:26 +08:00
26b1ef428a [branch-2.1](doris compose) fix docker start failed (#36534) 2024-06-20 20:14:17 +08:00
838af13001 [fix](auth)ldap set passwd need forward to master (#36436) (#36598)
pick from master: #36436
2024-06-20 18:35:37 +08:00
3ee259fc00 [branch-2.1][fix](jdbc catalog) fix jdbc mysql client match jsonb type (#36180)
bp #36177
2024-06-20 18:33:27 +08:00
ac0f6e75d2 [bugfix](iceberg)Read error when timestamp does not have time zone for 2.1 (#36435)
bp: #36141
2024-06-20 18:32:31 +08:00
22d37ba3fe [fix](auth)Auth support case insensitive (#36381) (#36557)
pick from: #36381
2024-06-20 18:31:30 +08:00
f7f7b2b738 [Enhancement](multi-catalog) Add more error msgs for wrong data types in orc and parquet reader. (#36580)
Backport #36417
2024-06-20 18:10:25 +08:00
fbcf63e1f5 [cherry-pick] (branch-2.1)fix variant index (#36577)
pick from master #36163
2024-06-20 17:57:26 +08:00
64a94e883d [fix](nereids)NullSafeEqualToEqual rule should keep <=> unchanged if it has none-literal child (#36523)
pick from master #36521

convert:
expr <=> null to expr is null
null <=> null to true
null <=> 1 to false
literal <=> literal to literal = literal ( 1 <=> 2 to 1 = 2 )
others are unchanged.
2024-06-20 17:55:36 +08:00
bd47d5a681 [branch-2.1](auto-partition) Fix auto partition load failure in multi replica (#36586)
this pr
1. picked #35630, which was reverted #36098 before.
2. picked #36344 from master

these two pr fixed existing bug about auto partition load.

---------

Co-authored-by: Kaijie Chen <ckj@apache.org>
2024-06-20 17:51:18 +08:00
6df1a9ab75 [branch-2.1](auto-partition) fix auto partition expr change unexpected (#36345) (#36514)
pick #36345
2024-06-20 17:50:31 +08:00
cbaff8a700 [fix](nereids)change the decimal's precision and scale for cast(xx as decimal) (#36540)
pick from master #36316

expression cast( xx as decimal )'s datatype maybe decimalv3 or decimalv2
depending on enable_decimal_conversion value in fe conf file. if
enable_decimal_conversion is true, the datatype is decimalv3(9, 0), but
the datatype was decimalv3(38, 9) in 2.0 releases. So this pr change the
datatype same as 2.0 releases to keep the behavior consistent.
2024-06-20 17:46:11 +08:00
1a242b8ae0 [cherry-pick](branch2.1) fix week/yearweek function get wrong result (#36538)
## Proposed changes
cherry-pick from master #36000 #36159
2024-06-20 15:48:19 +08:00
88e02c836d [Fix]Fix insert select missing audit log when connect follower FE (#36481)
## Proposed changes

pick #36472
2024-06-20 15:16:16 +08:00
c5bb0e3a21 [bug](prepared statement) fix prepared statement throw exception when inserting null value (#36484)
## Proposed changes

bp #36426

<!--Describe your changes.-->
2024-06-20 11:31:59 +08:00
9c1f34359d [fix](point query) should check it is Slot before check it is DELETE_SIGN (#36566)
pick from master #36564

introduced by #36443
2024-06-20 10:29:21 +08:00
7b36e81b7a [fix](split) FileSystemCacheKey are always different in overload equals (#36431)
bp: #36432
## Proposed changes

## Fixed Bugs introduced from #34307
1. `FileSystemCacheKey.equals()` compares properties by `==`, resulting
in creating new file system in each partition
2. `dfsFileSystem` is not synchronized, resulting in creating more file
systems than need.
3. `jobConf.iterator()` will produce more than 2000 pairs of key-value
2024-06-20 10:08:05 +08:00
dabd27edd2 [opt](inverted index) performance optimization for need_read_data in compound #35346 #36292 (#36404)
pick from master
https://github.com/apache/doris/pull/35346
https://github.com/apache/doris/pull/36292
2024-06-20 08:43:16 +08:00
0be5331b28 [Fix](Variant) fix variant schema change may cause invalid block schema and write missing blocks #36317 (#36536) 2024-06-19 19:09:16 +08:00
5b7d93df5e [Pick](Variant) pick 2 PRs to correct tmp column name to go fast execute #36277 #36313 (#36527) 2024-06-19 19:07:47 +08:00
c1f15f7e4c [fix](catalog) fix wrong check when using "use_meta_cache=true" (#36533)
bp #36530
2024-06-19 18:03:03 +08:00
8d5b621021 [improvement](inverted index) Change inverted index field_name from column_name to id in format v2 #36470 (#36516)
pick from master #36470
2024-06-19 17:29:26 +08:00
f59dc4fb37 [opt](split) generate and get split batch concurrently (#36044)
bp #36045, and turn on batch split, which is turn off in #36109
Generate and get split batch concurrently.
`SplitSource.getNextBatch` remove the synchronization, and make each get their splits concurrently, and `SplitAssignment` generates splits asynchronously.
2024-06-19 16:16:02 +08:00
9c896efe0b [fix](race) fix access colocate group ids race #36444 (#36501)
cherry pick from #36444
2024-06-19 15:38:50 +08:00
349b943e12 [opt](Nereids) Optimize Join Penalty Calculation Based on Build Side Data Volume (#36107)
pick from master #35773

This PR introduces an optimization that adjusts the penalty applied
during join operations based on the volume of data on the build side.
Specifically, when the number of rows and width of the tables being
joined are equal, the materialization costs are now considered more
accurately. The update ensures that joins with a larger dataset on the
build side incur a higher penalty, improving overall query performance
and resource allocation.
2024-06-19 14:49:09 +08:00