Commit Graph

33 Commits

Author SHA1 Message Date
fefbde8927 [log](move-memtable) improve logs in vtablet_writer_v2 and load_stream (#33103) 2024-04-12 15:09:25 +08:00
cf7595d423 [opt](memory) Optimize mem tracker accuracy (#32039) (#33140) 2024-04-10 11:42:19 +08:00
49d17f2be2 [fix](move-memtable) fix potential duplicate of TabletStream profile (#30397) 2024-01-30 15:30:14 +08:00
1a51d04cb8 [fix](move-memtable) fix schema use-after-free in delta writer v2 (#30254) 2024-01-24 10:00:25 +08:00
fd6ec97731 [fix](injection) don't disturb CLOSE_LOAD message in LoadStream (#30097) 2024-01-19 15:48:56 +08:00
41875a0bf5 [fix](move-memtable) check segment id in add_segment (#29898) 2024-01-16 18:30:23 +08:00
bfd23e30f6 [improve](load) handle EAGAIN in load stream (#29437) 2024-01-04 23:02:11 +08:00
96acef908a [fix](move-memtable) check eos when close stream (#29547) 2024-01-04 22:56:52 +08:00
329d57fdd7 [regression](move-memtable) test LoadStream on_idle_timeout (#29354)
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
2024-01-03 14:07:51 +08:00
1e8bb75182 [improve](move-memtable) add log on idle timeout (#29438) 2024-01-03 11:26:26 +08:00
243b075eef [improve](move-memtable) report failed tablets with status (#29351) 2024-01-02 20:24:53 +08:00
cc40cdf2d4 [improve](load) don't limit add_segment running count in load stream (#29332)
Co-authored-by: Yongqiang YANG <98214048+dataroaring@users.noreply.github.com>
2023-12-31 16:47:46 +08:00
e62857e5a4 [improve](move-memtable) return error when back pressure cost too much time (#29327) 2023-12-30 01:31:05 +08:00
b31494b18c [test](regression) add fault injection cases for LoadStream (#29101)
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
2023-12-28 16:16:26 +08:00
4a60d01dc7 [improve](move-memtable) increase load_stream_flush_token_max_tasks (#29011) 2023-12-26 17:08:49 +08:00
3b830f89a7 [improve](move-memtable) avoid using heavy work pool during append data (#28745) 2023-12-22 22:51:30 +08:00
18c9ebce95 [improve](move-memtable) tweak load stream flush token num and max tasks (#28884) 2023-12-22 20:08:47 +08:00
db523dafcb [improve](move-memtable) limit task num in load stream flush token (#28748) 2023-12-21 12:19:58 +08:00
4e86f9bab5 [improve](move-memtable) include and check offset when append data (#28159) 2023-12-09 16:21:36 +08:00
9c63dfd692 [improve](move-memtable) support variant in move-memtable (#28084)
* [improve](move-memtable) support variant in move-memtable

Pass the flush schema to the destination end of the sink.
2023-12-07 16:41:38 +08:00
c5f00fa746 [fix](move-memtable) temporary disable load stream profile (#28029) 2023-12-07 12:05:29 +08:00
75d0beb8cc [fix](move-memtable) only report load stream profile in the end (#27983) 2023-12-05 11:30:54 +08:00
b457856bd2 [chore](be) remove bthread scanner related codes (#27417) 2023-11-23 15:18:49 +08:00
b79f5d77f1 [improve](move-memtable) improve logging messages (#27443) 2023-11-23 11:46:29 +08:00
f6a9914bc7 [feature](move-memtable) support auto partition in sink v2 (#26914) 2023-11-14 11:39:44 +08:00
58bf79f79e [fix](move-memtable) pass load stream num to backends (#26198) 2023-11-08 16:16:33 +08:00
4995ca8fba [fix](move-memtable) ensure segment is flushed before add segment (#26522) 2023-11-07 22:42:16 +08:00
32b36d3c9c [refactor](move-memtable) rename proto OpenStreamSink to OpenLoadStream (#26527) 2023-11-07 22:41:20 +08:00
69c3e08699 [improve](load) set load id in signal handler (#25813) 2023-10-24 21:58:39 +08:00
b9ddcbf729 [feature](merge-cloud) Rewrite code related to IOContext (#24269) 2023-09-15 19:57:58 +08:00
a542f107db [feature](move-memtable) buffer messages in load stream stub (#23721) 2023-09-02 13:42:34 +08:00
14310ad30b [improvement](move-memtable) wait StreamClose from remote (#23605)
* [fix](move-memtable) wait StreamClose from remote
2023-08-30 18:03:36 +08:00
98d0a2f6c1 [feature](move-memtable)[3/7] add load stream manager and rpc service (#23415)
Co-authored-by: zhengyu <freeman.zhang1992@gmail.com>
Co-authored-by: Yongqiang YANG <dataroaring@gmail.com>
Co-authored-by: laihui <1353307710@qq.com>
2023-08-25 00:08:04 +08:00