Commit Graph

2758 Commits

Author SHA1 Message Date
cbcb81f381 [FIX](complextype)fix compare_at base function support nested types (#29297) 2024-01-06 12:05:43 +08:00
7402fee1fc [feature](function) support ip function ipv6_string_to_num(_or_default, _or_null), inet6_aton (#28361) 2024-01-05 19:24:45 +08:00
eea6219920 [improvement](group commit) Add an option of enable relay wal on all regression test (#28757) 2024-01-05 18:27:31 +08:00
46ccbebfb0 [improve](function) get result from constant_col when expr is const (#29403) 2024-01-05 11:26:17 +08:00
8c40f04f2b [Opt](parquet-reader) Opt ColumnSelectVector::set_run_length_null_map() in parquet reader. (#28954) (#29527) 2024-01-05 11:13:40 +08:00
08f2b83b81 [fix](profile) Fix row distribution timer in loadv1 (#29495)
Fix row distribution timer in loadv1
2024-01-05 10:02:18 +08:00
96acef908a [fix](move-memtable) check eos when close stream (#29547) 2024-01-04 22:56:52 +08:00
f28dbc702c [bugfix](scanner done) should not set process status to query context (#29512)
Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-01-04 15:18:10 +08:00
bfe65565d8 [feature](paimon)support native reader (#29339)
Support native reader fro paimon.

Upgrade paimon 0.5 to 0.6 : apache/doris-shade#32
2024-01-04 14:31:48 +08:00
Pxl
d8a08dad90 [Bug](mark-join) fix wrong result on mark join + other conjunct (#29321)
fix wrong result on mark join + other conjunct
2024-01-04 11:58:39 +08:00
bd8113f424 [bugfix](scannerscheduler) should minus num_of_scanners before check should schedule #28926 (#29331)
---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-01-03 20:47:35 +08:00
c84cd30223 [pipelineX](fix) Fix query cancel timeout (#29460)
There are 2 potential reasons to cancel pipelineX query timeout.

Cancel fragment context first and set ready to execute will set cancel flag to false.
Dead lock.
2024-01-03 20:29:04 +08:00
329d57fdd7 [regression](move-memtable) test LoadStream on_idle_timeout (#29354)
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
2024-01-03 14:07:51 +08:00
1fbbff32b2 [fix](pipelinex) coredump caused by VRuntimeFilterSlots::_is_global was not set (#29446) 2024-01-03 12:40:41 +08:00
c0db8533af [fix](load) fix single replica load with auto partition 2024-01-03 11:53:09 +08:00
xy
fab1a627fc [fix](scan) _insert_data_normal should catch exception when BlockReader::_unique_key_next_block (#29426)
Co-authored-by: xingying01 <xingying01@corp.netease.com>
2024-01-03 11:44:02 +08:00
14e7eb7624 [Opt](rf) Opt broadcast join remote runtime filter merge and wait (#29439) 2024-01-03 11:21:28 +08:00
Pxl
358995e4ac [Chore](sort) fix block used after it was moved(#29416)
/mnt/disk2/pengyu/codebase/apache/doris/be/src/vec/common/sort/heap_sorter.cpp:91:23: error: 'tmp_block' used after it was moved [bugprone-use-after-move,-warnings-as-errors]
2024-01-03 10:02:35 +08:00
f54f79515c [Bug](fix) str_to_date "" should be null (#29402) 2024-01-03 08:25:22 +08:00
69a01e0cf5 [improve](move-memtable) skip load stream stub close wait when cancel (#29427) 2024-01-02 23:35:50 +08:00
5db496d844 [Improve](Variant) make output stable (#29389) 2024-01-02 20:29:17 +08:00
9490d5e9a2 [Debug](Variant) sanitize variant in write_column_to_mysql (#29380) 2024-01-02 20:28:59 +08:00
243b075eef [improve](move-memtable) report failed tablets with status (#29351) 2024-01-02 20:24:53 +08:00
b3f69211b7 [refactor](wal) move group commit load content length to runtime state (#29188) 2024-01-02 15:53:38 +08:00
706463781c [refactor](group commit) refactor group commit wal code (#29375) 2024-01-02 15:52:03 +08:00
2ed122b787 [improvement](task exec context) add parent class HasTaskExecutionCtx to own the task ctx (#29388)
---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-01-02 15:28:27 +08:00
b7487430da Revert "[improve](move-memtable) cancel load rapidly when stream close wait (#29322)" (#29371)
This reverts commit bbf58c5aa42d40e66bc6ccc9ed91a4fcb4bdfff7.
2024-01-02 11:32:14 +08:00
cec69f4cbd [Enhancement](wal) Add timout for wal memory back pressure (#29178) 2024-01-02 11:02:17 +08:00
738abac9ed [minor](context) duplicate query context in fragment ctx (#29364)
---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-01-01 22:08:23 +08:00
b07ce175de Revert "[fix](move-memtable) only check missing tablets when commit i… (#29352)
* Revert "[fix](move-memtable) only check missing tablets when commit info is not empty (#29326)"

This reverts commit 4634c723d7fb86f434060c0e6fa6168d92630cb9.

* Revert "[fix](move-memtable) check missing tablets before commit (#29223)"

This reverts commit fb0ed8c253f5241c23c71fb88f031604b0bf5a1f.
2023-12-31 22:25:25 +08:00
bbf58c5aa4 [improve](move-memtable) cancel load rapidly when stream close wait (#29322) 2023-12-31 16:26:41 +08:00
4634c723d7 [fix](move-memtable) only check missing tablets when commit info is not empty (#29326)
Some sink may not get any success tablets, because load stream will only report them at the last CLOSE_LOAD.
For these sinks (no commit info), we should not check missing tablets since it's expected to be empty.
2023-12-31 13:50:13 +08:00
fcc4cfb900 [Fix](Variant) add more info before crash in serialization (#29344) 2023-12-31 11:17:36 +08:00
7623b5cc31 [cleanup](move-memtable) remove namespace stream_load (#27441) 2023-12-30 20:08:23 +08:00
03901b9a7a [enhancement](group_commit): refector relay wal code (#29183) 2023-12-30 12:59:46 +08:00
e7d67e9411 [fix](be) resolves some unused-raii and used-after-moved issues (#29285) 2023-12-30 12:14:49 +08:00
51cb15d032 [improve](move-memtable) cancel load immediately when back pressure in delta writer v2 (#29280) 2023-12-30 10:45:06 +08:00
5bc72bda78 [improve](move-memtable) add fault injection in writer v2 (#29177) 2023-12-30 01:32:26 +08:00
41d907912a [improve](move-memtable) add fault injection in load stream stub (#29105) 2023-12-30 01:29:28 +08:00
411f819306 [Opt](join) opt the join copy column from build side (#29007) 2023-12-29 14:39:33 +08:00
b9572f9de0 [pipelineX](fix) Fix pip scanner context bug (#29229) 2023-12-29 13:24:39 +08:00
a525d5c5a3 [refactor](decimal) change type name Decimal128 to Decimal128V2, Decimal128I to Decimal128V3 to avoid confusion (#29265)
change type name Decimal128 to Decimal128V2, Decimal128I to Decimal128V3 to avoid confusion
2023-12-29 10:11:44 +08:00
269c1b189d [improve](vtablet_writer) check runtime state is cancel when back pressure (#29260) 2023-12-29 09:11:24 +08:00
fb0ed8c253 [fix](move-memtable) check missing tablets before commit (#29223) 2023-12-29 00:33:58 +08:00
9be0f04506 (improv)[group commit] refactor some group commit code (#29180) 2023-12-29 00:26:10 +08:00
a90304c208 [fix](parquet) complex type in parquet is case sensitive (#29245)
Change name of complex type in parquet to case-insensitive. Otherwise, uppercase column names of complex types will return null.
2023-12-28 22:43:11 +08:00
ffd178f5ff [feat](pipelinex) support parallel scan on pipeline x engine (#29070)
* [feat](pipelinex) support parallel scan on pipeline x engine

* make parallel scan be independent of shared scan
2023-12-28 21:29:07 +08:00
5129ab5738 [fix](decimalv2) fix decimalv2 agg errors (#29246) 2023-12-28 21:17:16 +08:00
fe93a8f1d0 [cleanup](move-memtable) remove unused log in load stream stub (#29084) 2023-12-28 14:39:10 +08:00
3dc3e81734 [Improvement](datatype) Update Parser for IPv4/v6 data types (#29044)
Transforming from parsing std:: string to parsing char * to accelerate the parsing of ipv4/v6 data types.
2023-12-28 11:00:38 +08:00