database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

[improvememt](file-cache) increase virtual node number to make file cache more even (#24143)

Browse Source 
The origin virtual number is Math.max(Math.min(512 / backends.size(), 32), 2);, which is too small,
causing uneven cache distribution when enabling file cache.
This commit is contained in:
Mingyu Chen
2023-09-10 19:56:53 +08:00
committed by GitHub
parent 102abff071
commit 1df2e4454f
2 changed files with 9 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
fe/fe-core/src/main/java/org/apache/doris/planner/external/FederationBackendPolicy.java vendored
View File

@ -96,9 +96,8 @@ public class FederationBackendPolicy {
throw new UserException("No available backends");
}
backendMap.putAll(backends.stream().collect(Collectors.groupingBy(Backend::getHost)));
int virtualNumber = Math.max(Math.min(512 / backends.size(), 32), 2);
consistentHash = new ConsistentHash<>(Hashing.murmur3_128(), new ScanRangeHash(),
new BackendHash(), backends, virtualNumber);
new BackendHash(), backends, Config.virtual_node_number);
}
public Backend getNextBe() {