database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

Add ORC help doc (#3041)

Browse Source
This commit is contained in:
ZHAO Chun
2020-03-05 12:44:47 +08:00
committed by GitHub
parent c731c8b9bc
commit cd7207c869
2 changed files with 29 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
docs/documentation/cn/administrator-guide/load-data/broker-load-manual.md
View File

@ -490,5 +490,19 @@ LoadFinishTime: 2019-07-27 11:50:16
* 导入报错：`failed to send batch` 或 `TabletWriter add batch with unknown id`
请参照 [导入手册](./load-manual.md) 中 **通用系统配置** 中 **BE 配置**，适当修改 `tablet_writer_rpc_timeout_sec` 和 `streaming_load_rpc_max_alive_time_sec`。
* 导入报错：`LOAD_RUN_FAIL; msg:Invalid Column Name:xxx`
如果是PARQUET或者ORC格式的数据,需要再文件头的列名与doris表中的列名一致，如 :
```
(tmp_c1,tmp_c2)
SET
(
id=tmp_c2,
name=tmp_c1
)
```
代表获取在parquet或orc中以(tmp_c1, tmp_c2)为列名的列，映射到doris表中的(id, name)列。如果没有设置set, 则以column中的列作为映射。
注：如果使用某些hive版本直接生成的orc文件，orc文件中的表头并非hive meta数据，而是（_col0, _col1, _col2, ...）, 可能导致Invalid Column Name错误，那么则需要使用set进行映射

15
docs/documentation/en/administrator-guide/load-data/broker-load-manual_EN.md
View File

@ -471,3 +471,18 @@ Cluster situation: The number of BEs in the cluster is about 3, and the Broker n
* failed with ：`failed to send batch` or `TabletWriter add batch with unknown id`
Refer to **General System Configuration** in **BE Configuration** in the Import Manual (./load-manual.md), and modify `tablet_writer_rpc_timeout_sec` and `streaming_load_rpc_max_alive_time_sec` appropriately.
* failed with : `LOAD_RUN_FAIL; msg: Invalid Column Name: xxx`
    
     If it is PARQUET or ORC format data, you need to keep the column names in the file header consistent with the column names in the doris table, such as:
     `` `
     (tmp_c1, tmp_c2)
     SET
     (
         id = tmp_c2,
         name = tmp_c1
     )
     `` `
     Represents getting the column with (tmp_c1, tmp_c2) as the column name in parquet or orc, which is mapped to the (id, name) column in the doris table. If set is not set, the column names in the column are used as the mapping relationship.
     Note: If the orc file directly generated by some hive versions is used, the table header in the orc file is not the column name in the hive meta, but (_col0, _col1, _col2, ...), which may cause the Invalid Column Name error, then You need to use set for mapping.