branch-2.1: [fix](multi-catalog) Fix bug: "Can not create a Path from an empty string" (#49382) (#49641)
### What problem does this PR solve?
Problem Summary:
In HiveMetaStoreCache, the function FileInputFormat.setInputPaths is
used to set input paths. However, this function splits paths using
commas, which is not the expected behavior. As a result, when partition
values contain commas, it leads to incorrect path parsing and potential
errors.
```java
public static void setInputPaths(JobConf conf, String org.apache.hadoop.shaded.com.aSeparatedPaths) {
setInputPaths(conf, StringUtils.stringToPath(
getPathStrings(org.apache.hadoop.shaded.com.aSeparatedPaths)));
}
```
To prevent FileInputFormat.setInputPaths from splitting paths by commas,
we use another overloaded version of the method. Instead of passing a
comma-separated string, we explicitly pass a Path object, ensuring that
partition values containing commas are handled correctly.
```java
public static void setInputPaths(JobConf conf, Path... inputPaths) {
Path path = new Path(conf.getWorkingDirectory(), inputPaths[0]);
StringBuffer str = new StringBuffer(StringUtils.escapeString(path.toString()));
for(int i = 1; i < inputPaths.length;i++) {
str.append(StringUtils.COMMA_STR);
path = new Path(conf.getWorkingDirectory(), inputPaths[i]);
str.append(StringUtils.escapeString(path.toString()));
}
conf.set(org.apache.hadoop.shaded.org.apache.hadoop.mapreduce.lib.input.
FileInputFormat.INPUT_DIR, str.toString());
}
```
### Release note
None
This commit is contained in:
@ -404,7 +404,8 @@ public class HiveMetaStoreCache {
|
||||
} catch (Exception e) {
|
||||
LOG.warn("unknown scheme in path: " + finalLocation, e);
|
||||
}
|
||||
FileInputFormat.setInputPaths(jobConf, finalLocation.get());
|
||||
// NOTICE: the setInputPaths has 2 overloads, the 2nd arg should be Path not String
|
||||
FileInputFormat.setInputPaths(jobConf, finalLocation.getPath());
|
||||
try {
|
||||
FileCacheValue result = getFileCache(finalLocation.get(), key.inputFormat, jobConf,
|
||||
key.getPartitionValues(), key.bindBrokerName);
|
||||
|
||||
Reference in New Issue
Block a user