[enhancement](load) Increase batch size of node channel to improve import performance (#13912)

This commit is contained in:
Xin Liao
2022-11-11 18:05:36 +08:00
committed by GitHub
parent 2e29b15c6a
commit 43f80e2633

View File

@ -68,6 +68,12 @@ Status VNodeChannel::init(RuntimeState* state) {
_cur_add_block_request.set_eos(false);
_name = fmt::format("VNodeChannel[{}-{}]", _index_channel->_index_id, _node_id);
// The node channel will send _batch_size rows of data each rpc. When the
// number of tablets is large, the number of data rows received by each
// tablet is small, TabletsChannel need to traverse each tablet for import.
// so the import performance is poor. Therefore, we set _batch_size to
// a relatively large value to improve the import performance.
_batch_size = std::max(_batch_size, 8192);
return Status::OK();
}