Commit Graph

22 Commits

Author SHA1 Message Date
cc40cdf2d4 [improve](load) don't limit add_segment running count in load stream (#29332)
Co-authored-by: Yongqiang YANG <98214048+dataroaring@users.noreply.github.com>
2023-12-31 16:47:46 +08:00
e62857e5a4 [improve](move-memtable) return error when back pressure cost too much time (#29327) 2023-12-30 01:31:05 +08:00
b31494b18c [test](regression) add fault injection cases for LoadStream (#29101)
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
2023-12-28 16:16:26 +08:00
4a60d01dc7 [improve](move-memtable) increase load_stream_flush_token_max_tasks (#29011) 2023-12-26 17:08:49 +08:00
3b830f89a7 [improve](move-memtable) avoid using heavy work pool during append data (#28745) 2023-12-22 22:51:30 +08:00
18c9ebce95 [improve](move-memtable) tweak load stream flush token num and max tasks (#28884) 2023-12-22 20:08:47 +08:00
db523dafcb [improve](move-memtable) limit task num in load stream flush token (#28748) 2023-12-21 12:19:58 +08:00
4e86f9bab5 [improve](move-memtable) include and check offset when append data (#28159) 2023-12-09 16:21:36 +08:00
9c63dfd692 [improve](move-memtable) support variant in move-memtable (#28084)
* [improve](move-memtable) support variant in move-memtable

Pass the flush schema to the destination end of the sink.
2023-12-07 16:41:38 +08:00
c5f00fa746 [fix](move-memtable) temporary disable load stream profile (#28029) 2023-12-07 12:05:29 +08:00
75d0beb8cc [fix](move-memtable) only report load stream profile in the end (#27983) 2023-12-05 11:30:54 +08:00
b457856bd2 [chore](be) remove bthread scanner related codes (#27417) 2023-11-23 15:18:49 +08:00
b79f5d77f1 [improve](move-memtable) improve logging messages (#27443) 2023-11-23 11:46:29 +08:00
f6a9914bc7 [feature](move-memtable) support auto partition in sink v2 (#26914) 2023-11-14 11:39:44 +08:00
58bf79f79e [fix](move-memtable) pass load stream num to backends (#26198) 2023-11-08 16:16:33 +08:00
4995ca8fba [fix](move-memtable) ensure segment is flushed before add segment (#26522) 2023-11-07 22:42:16 +08:00
32b36d3c9c [refactor](move-memtable) rename proto OpenStreamSink to OpenLoadStream (#26527) 2023-11-07 22:41:20 +08:00
69c3e08699 [improve](load) set load id in signal handler (#25813) 2023-10-24 21:58:39 +08:00
b9ddcbf729 [feature](merge-cloud) Rewrite code related to IOContext (#24269) 2023-09-15 19:57:58 +08:00
a542f107db [feature](move-memtable) buffer messages in load stream stub (#23721) 2023-09-02 13:42:34 +08:00
14310ad30b [improvement](move-memtable) wait StreamClose from remote (#23605)
* [fix](move-memtable) wait StreamClose from remote
2023-08-30 18:03:36 +08:00
98d0a2f6c1 [feature](move-memtable)[3/7] add load stream manager and rpc service (#23415)
Co-authored-by: zhengyu <freeman.zhang1992@gmail.com>
Co-authored-by: Yongqiang YANG <dataroaring@gmail.com>
Co-authored-by: laihui <1353307710@qq.com>
2023-08-25 00:08:04 +08:00