database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

Set the load channel's timeout to be the same as the load job's timeout (#2405)

Browse Source 
[Load] 

When performing a long-time load job, the following errors may occur. Causes the load to fail.

load channel manager add batch with unknown load id: xxx

There is a case of this error because Doris opened an unrelated channel during the load
process. This channel will not receive any data during the entire load process. Therefore,
after a fixed timeout, the channel will be released.

And after the entire load job is completed, it will try to close all open channels. When it try to
close this channel, it will find that the channel no longer exists and an error is reported.

This CL will pass the timeout of load job to the load channel, so that the timeout of load channels
will be same as load job's.
This commit is contained in:
Mingyu Chen
2019-12-06 21:51:00 +08:00
committed by ZHAO Chun
parent 55d64e3be8
commit a3b7cf484b
18 changed files with 70 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
fe/src/test/java/org/apache/doris/planner/OlapTableSinkTest.java
View File

@ -100,7 +100,7 @@ public class OlapTableSinkTest {
}};
OlapTableSink sink = new OlapTableSink(dstTable, tuple, "");
sink.init(new TUniqueId(1, 2), 3, 4);
sink.init(new TUniqueId(1, 2), 3, 4, 1000);
sink.finalize();
LOG.info("sink is {}", sink.toThrift());
LOG.info("{}", sink.getExplainString("", TExplainLevel.NORMAL));
@ -136,7 +136,7 @@ public class OlapTableSinkTest {
}};
OlapTableSink sink = new OlapTableSink(dstTable, tuple, "p1");
sink.init(new TUniqueId(1, 2), 3, 4);
sink.init(new TUniqueId(1, 2), 3, 4, 1000);
try {
sink.finalize();
} catch (UserException e) {
@ -158,7 +158,7 @@ public class OlapTableSinkTest {
}};
OlapTableSink sink = new OlapTableSink(dstTable, tuple, "p3");
sink.init(new TUniqueId(1, 2), 3, 4);
sink.init(new TUniqueId(1, 2), 3, 4, 1000);
sink.finalize();
LOG.info("sink is {}", sink.toThrift());
LOG.info("{}", sink.getExplainString("", TExplainLevel.NORMAL));
@ -175,7 +175,7 @@ public class OlapTableSinkTest {
}};
OlapTableSink sink = new OlapTableSink(dstTable, tuple, "p1");
sink.init(new TUniqueId(1, 2), 3, 4);
sink.init(new TUniqueId(1, 2), 3, 4, 1000);
sink.finalize();
LOG.info("sink is {}", sink.toThrift());
LOG.info("{}", sink.getExplainString("", TExplainLevel.NORMAL));