Commit Graph

2774 Commits

Author SHA1 Message Date
7287c0ca15 [Opt](exec)(multi-catalog) Opt date type reading. (#29571) 2024-01-12 11:48:39 +08:00
463a7ab212 [Performance](exec) opt the exchange performance (#29579) 2024-01-12 11:46:29 +08:00
48f58510a8 [refactor](tabletwriter) make tablet writer's rpc callback safe, could exit any time (#29684)
---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-01-12 11:46:29 +08:00
abb7640d37 [debug](timeout) add more log in scanner ctx to find timeout problem #29704
Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-01-12 11:44:21 +08:00
be56bf06cf [feature](function) support ip function named is_ip_address_in_range(addr, cidr) (#29681) 2024-01-12 11:44:21 +08:00
8fc9c18c85 [improvement](jdbc catalog) Put the jdbc connection pool parameters into catalog properties (#29195) 2024-01-12 11:40:28 +08:00
9ef4e49307 [bugfix](scannerdeadloop) there is a dead loop in scanner ctx (#29794)
Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-01-11 16:47:54 +08:00
767de7afe8 Revert "[feature](pipelineX) control exchange sink by memory usage (#28814)" (#29652)
This reverts commit e326ebb63e4e07d8ee6595561ab19dc5d411f592.
2024-01-08 21:48:51 +08:00
Pxl
e556536de1 [Refactor](join) split SetHashTableVariants out from HashTableVariants (#29519)
split SetHashTableVariants out from HashTableVariants
2024-01-08 10:37:00 +08:00
c58d18147b [refactor](resultwriter) rename append_block to write method is more meaningful (#29635)
---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-01-08 00:00:33 +08:00
c497f749ce [debug](timeout) debug select timeout (#29627)
---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-01-07 19:54:02 +08:00
0b731800a0 [enhancement](group_commit) refector wal manager code (#29560) 2024-01-07 18:54:41 +08:00
eb4c389b0b [feature](function) support ip functions isipv4string and isipv6string (#28556) 2024-01-07 13:03:11 +08:00
2adb0fcc50 [opt](hive) support orc generated from hive 1.x for all file scan node (#28806) 2024-01-06 17:33:16 +08:00
2c888667ed [improvement](function) standardize some ip functions' signatures #29614
The signatures of functions in these PRs should be more standard:
#27342,
#25510,
#20936,
including the following:
ipv4numtostring,
ipv4stringtonum,
ipv4stringtonumordefault,
ipv4stringtonumornull,
ipv6numtostring.

This PR will add necessary underscores between the words of each of them,
like changing ipv4numtostring to ipv4_num_to_string.
2024-01-06 16:16:38 +08:00
5789b7e380 [fix](jin) add datetimev2 precision (#29528) 2024-01-06 13:35:26 +08:00
cbcb81f381 [FIX](complextype)fix compare_at base function support nested types (#29297) 2024-01-06 12:05:43 +08:00
7402fee1fc [feature](function) support ip function ipv6_string_to_num(_or_default, _or_null), inet6_aton (#28361) 2024-01-05 19:24:45 +08:00
eea6219920 [improvement](group commit) Add an option of enable relay wal on all regression test (#28757) 2024-01-05 18:27:31 +08:00
46ccbebfb0 [improve](function) get result from constant_col when expr is const (#29403) 2024-01-05 11:26:17 +08:00
8c40f04f2b [Opt](parquet-reader) Opt ColumnSelectVector::set_run_length_null_map() in parquet reader. (#28954) (#29527) 2024-01-05 11:13:40 +08:00
08f2b83b81 [fix](profile) Fix row distribution timer in loadv1 (#29495)
Fix row distribution timer in loadv1
2024-01-05 10:02:18 +08:00
96acef908a [fix](move-memtable) check eos when close stream (#29547) 2024-01-04 22:56:52 +08:00
f28dbc702c [bugfix](scanner done) should not set process status to query context (#29512)
Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-01-04 15:18:10 +08:00
bfe65565d8 [feature](paimon)support native reader (#29339)
Support native reader fro paimon.

Upgrade paimon 0.5 to 0.6 : apache/doris-shade#32
2024-01-04 14:31:48 +08:00
Pxl
d8a08dad90 [Bug](mark-join) fix wrong result on mark join + other conjunct (#29321)
fix wrong result on mark join + other conjunct
2024-01-04 11:58:39 +08:00
bd8113f424 [bugfix](scannerscheduler) should minus num_of_scanners before check should schedule #28926 (#29331)
---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-01-03 20:47:35 +08:00
c84cd30223 [pipelineX](fix) Fix query cancel timeout (#29460)
There are 2 potential reasons to cancel pipelineX query timeout.

Cancel fragment context first and set ready to execute will set cancel flag to false.
Dead lock.
2024-01-03 20:29:04 +08:00
329d57fdd7 [regression](move-memtable) test LoadStream on_idle_timeout (#29354)
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
2024-01-03 14:07:51 +08:00
1fbbff32b2 [fix](pipelinex) coredump caused by VRuntimeFilterSlots::_is_global was not set (#29446) 2024-01-03 12:40:41 +08:00
c0db8533af [fix](load) fix single replica load with auto partition 2024-01-03 11:53:09 +08:00
xy
fab1a627fc [fix](scan) _insert_data_normal should catch exception when BlockReader::_unique_key_next_block (#29426)
Co-authored-by: xingying01 <xingying01@corp.netease.com>
2024-01-03 11:44:02 +08:00
14e7eb7624 [Opt](rf) Opt broadcast join remote runtime filter merge and wait (#29439) 2024-01-03 11:21:28 +08:00
Pxl
358995e4ac [Chore](sort) fix block used after it was moved(#29416)
/mnt/disk2/pengyu/codebase/apache/doris/be/src/vec/common/sort/heap_sorter.cpp:91:23: error: 'tmp_block' used after it was moved [bugprone-use-after-move,-warnings-as-errors]
2024-01-03 10:02:35 +08:00
f54f79515c [Bug](fix) str_to_date "" should be null (#29402) 2024-01-03 08:25:22 +08:00
69a01e0cf5 [improve](move-memtable) skip load stream stub close wait when cancel (#29427) 2024-01-02 23:35:50 +08:00
5db496d844 [Improve](Variant) make output stable (#29389) 2024-01-02 20:29:17 +08:00
9490d5e9a2 [Debug](Variant) sanitize variant in write_column_to_mysql (#29380) 2024-01-02 20:28:59 +08:00
243b075eef [improve](move-memtable) report failed tablets with status (#29351) 2024-01-02 20:24:53 +08:00
b3f69211b7 [refactor](wal) move group commit load content length to runtime state (#29188) 2024-01-02 15:53:38 +08:00
706463781c [refactor](group commit) refactor group commit wal code (#29375) 2024-01-02 15:52:03 +08:00
2ed122b787 [improvement](task exec context) add parent class HasTaskExecutionCtx to own the task ctx (#29388)
---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-01-02 15:28:27 +08:00
b7487430da Revert "[improve](move-memtable) cancel load rapidly when stream close wait (#29322)" (#29371)
This reverts commit bbf58c5aa42d40e66bc6ccc9ed91a4fcb4bdfff7.
2024-01-02 11:32:14 +08:00
cec69f4cbd [Enhancement](wal) Add timout for wal memory back pressure (#29178) 2024-01-02 11:02:17 +08:00
738abac9ed [minor](context) duplicate query context in fragment ctx (#29364)
---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-01-01 22:08:23 +08:00
b07ce175de Revert "[fix](move-memtable) only check missing tablets when commit i… (#29352)
* Revert "[fix](move-memtable) only check missing tablets when commit info is not empty (#29326)"

This reverts commit 4634c723d7fb86f434060c0e6fa6168d92630cb9.

* Revert "[fix](move-memtable) check missing tablets before commit (#29223)"

This reverts commit fb0ed8c253f5241c23c71fb88f031604b0bf5a1f.
2023-12-31 22:25:25 +08:00
bbf58c5aa4 [improve](move-memtable) cancel load rapidly when stream close wait (#29322) 2023-12-31 16:26:41 +08:00
4634c723d7 [fix](move-memtable) only check missing tablets when commit info is not empty (#29326)
Some sink may not get any success tablets, because load stream will only report them at the last CLOSE_LOAD.
For these sinks (no commit info), we should not check missing tablets since it's expected to be empty.
2023-12-31 13:50:13 +08:00
fcc4cfb900 [Fix](Variant) add more info before crash in serialization (#29344) 2023-12-31 11:17:36 +08:00
7623b5cc31 [cleanup](move-memtable) remove namespace stream_load (#27441) 2023-12-30 20:08:23 +08:00