database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

[Bug] Fix bug that routine load blocked with TOO_MANY_TASKS error (#4861)

Browse Source 
When receiving empty msg from kafka, the load process will quit abnormally.
Fix #4860
This commit is contained in:
Mingyu Chen
2020-11-12 10:05:10 +08:00
committed by GitHub
parent 1810f10497
commit 796f44beac
1 changed files with 5 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
be/src/runtime/routine_load/data_consumer.cpp
View File

@ -193,7 +193,11 @@ Status KafkaDataConsumer::group_consume(
consumer_watch.stop();
switch (msg->err()) {
case RdKafka::ERR_NO_ERROR:
if (!queue->blocking_put(msg.get())) {
if (msg->len() == 0) {
// ignore msg with length 0.
// put empty msg into queue will cause the load process shutting down.
break;
} else if (!queue->blocking_put(msg.get())) {
// queue is shutdown
done = true;
} else {