database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

[fix](new-scan)Fix new scanner load job bugs (#12903)

Browse Source 
Fix bugs:
1. Fe need to send file format (e.g. parquet, orc ...) to be while processing load jobs using new scanner.
2. Try to get parquet file column type from SchemaElement.type before getting from Logical type and Converted type.
This commit is contained in:
Jibing-Li
2022-09-24 17:21:19 +08:00
committed by GitHub
parent 3bb920ba54
commit f1a64ea09f
3 changed files with 33 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
fe/fe-core/src/main/java/org/apache/doris/planner/external/LoadScanProvider.java vendored
View File

@ -93,6 +93,7 @@ public class LoadScanProvider implements FileScanProviderIf {
ctx.timezone = analyzer.getTimezone();
TFileScanRangeParams params = new TFileScanRangeParams();
params.format_type = formatType(fileGroupInfo.getFileGroup().getFileFormat(), "");
params.setStrictMode(fileGroupInfo.isStrictMode());
params.setProperties(fileGroupInfo.getBrokerDesc().getProperties());
if (fileGroupInfo.getBrokerDesc().getFileType() == TFileType.FILE_HDFS) {