database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

[config](checksum) Disable consistency checker by default (#9699)

Browse Source 
Disable by default because current checksum logic has some bugs.
And it will also bring some overhead.
This commit is contained in:
Mingyu Chen
2022-05-22 21:31:43 +08:00
committed by GitHub
parent ad4da4aa8f
commit d270f4f2d4
4 changed files with 22 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
docs/en/admin-manual/config/fe-config.md
View File

@ -1017,17 +1017,21 @@ IsMutable：true
MasterOnly：true
Consistency checker will run from *consistency_check_start_time* to *consistency_check_end_time*. Default is from 23:00 to 04:00
Consistency checker will run from *consistency_check_start_time* to *consistency_check_end_time*.
If the two times are the same, no consistency check will be triggered.
### consistency_check_end_time
Default：04
Default：23
IsMutable：true
MasterOnly：true
Consistency checker will run from *consistency_check_start_time* to *consistency_check_end_time*. Default is from 23:00 to 04:00
Consistency checker will run from *consistency_check_start_time* to *consistency_check_end_time*.
If the two times are the same, no consistency check will be triggered.
### export_tablet_num_per_task

10
docs/zh-CN/admin-manual/config/fe-config.md
View File

@ -1017,11 +1017,13 @@ colocote join PlanFragment instance 的 memory_limit = exec_mem_limit / min (que
一致性检查开始时间
一致性检查器将从 `consistency_check_start_time` 运行到 `consistency_check_end_time`。 默认为 23:00 至 04:00
一致性检查器将从 `consistency_check_start_time` 运行到 `consistency_check_end_time`。
如果两个时间相同，则不会触发一致性检查。
### `consistency_check_end_time`
默认值：04
默认值：23
是否可以动态配置：true
@ -1029,7 +1031,9 @@ colocote join PlanFragment instance 的 memory_limit = exec_mem_limit / min (que
一致性检查结束时间
一致性检查器将从 `consistency_check_start_time` 运行到 `consistency_check_end_time`。 默认为 23:00 至 04:00
一致性检查器将从 `consistency_check_start_time` 运行到 `consistency_check_end_time`。
如果两个时间相同，则不会触发一致性检查。
### `export_tablet_num_per_task`

7
fe/fe-core/src/main/java/org/apache/doris/common/Config.java
View File

@ -856,12 +856,15 @@ public class Config extends ConfigBase {
// Configurations for consistency check
/**
* Consistency checker will run from *consistency_check_start_time* to *consistency_check_end_time*.
* Default is from 23:00 to 04:00
* If start time == end time, the checker will stop scheduling.
* And default is disabled.
* TODO(cmy): Disable by default because current checksum logic has some bugs.
* And it will also bring some overhead.
*/
@ConfField(mutable = true, masterOnly = true)
public static String consistency_check_start_time = "23";
@ConfField(mutable = true, masterOnly = true)
public static String consistency_check_end_time = "4";
public static String consistency_check_end_time = "23";
/**
* Default timeout of a single consistency check task. Set long enough to fit your tablet size.
*/

4
fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java
View File

@ -98,6 +98,7 @@ import com.google.common.collect.Lists;
import com.google.common.collect.Maps;
import com.google.common.collect.Multiset;
import com.google.common.collect.Sets;
import org.apache.commons.collections.map.HashedMap;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
@ -640,7 +641,8 @@ public class Coordinator {
switch (code) {
case TIMEOUT:
throw new RpcException(pair.first.backend.getHost(), "send fragment timeout. backend id: "
+ pair.first.backend.getId());
+ pair.first.backend.getId() + " fragment: " +
DebugUtil.printId(pair.first.rpcParams.params.fragment_instance_id));
case THRIFT_RPC_ERROR:
SimpleScheduler.addToBlacklist(pair.first.backend.getId(), errMsg);
throw new RpcException(pair.first.backend.getHost(), "rpc failed");