database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

[doc](multi-catalog)huawei mrs event sync doc (#20466)

Browse Source
This commit is contained in:
zhangdong
2023-06-07 20:05:12 +08:00
committed by GitHub
parent 2db900b775
commit 809bb46518
2 changed files with 56 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

29
docs/en/docs/lakehouse/multi-catalog/multi-catalog.md
View File

@ -347,7 +347,7 @@ The automatic update feature involves the following parameters in fe.conf:
2. `hms_events_polling_interval_ms`: This specifies the interval between two readings, which is set to 10000 by default. (Unit: millisecond)
3. `hms_events_batch_size_per_rpc`: This specifies the maximum number of events that are read at a time, which is set to 500 by default.
To enable automatic update, you need to modify the hive-site.xml of HMS and then restart HMS and HiveServer2:
To enable automatic update(Excluding Huawei MRS), you need to modify the hive-site.xml of HMS and then restart HMS and HiveServer2:
```
<property>
@ -365,6 +365,33 @@ To enable automatic update, you need to modify the hive-site.xml of HMS and then
```
Huawei's MRS needs to change hivemetastore-site.xml and restart HMS and HiveServer2:
```
<property>
<name>metastore.transactional.event.listeners</name>
<value>org.apache.hive.hcatalog.listener.DbNotificationListener</value>
</property>
```
Note: Value is appended with commas separated from the original value, not overwritten.For example, the default configuration for MRS 3.1.0 is
```
<property>
<name>metastore.transactional.event.listeners</name>
<value>com.huawei.bigdata.hive.listener.TableKeyFileManagerListener,org.apache.hadoop.hive.metastore.listener.FileAclListener</value>
</property>
```
We need to change to
```
<property>
<name>metastore.transactional.event.listeners</name>
<value>com.huawei.bigdata.hive.listener.TableKeyFileManagerListener,org.apache.hadoop.hive.metastore.listener.FileAclListener,org.apache.hive.hcatalog.listener.DbNotificationListener</value>
</property>
```
> Note: To enable automatic update, whether for existing Catalogs or newly created Catalogs, all you need is to set `enable_hms_events_incremental_sync` to `true`, and then restart the FE node. You don't need to manually update the metadata before or after the restart.
<version since="dev">

29
docs/zh-CN/docs/lakehouse/multi-catalog/multi-catalog.md
View File

@ -347,7 +347,7 @@ Doris 的权限管理功能提供了对 Catalog 层级的扩展，具体可参
2. `hms_events_polling_interval_ms`: 读取 event 的间隔时间，默认值为 10000，单位：毫秒。
3. `hms_events_batch_size_per_rpc`: 每次读取 event 的最大数量，默认值为 500。
如果想使用该特性，需要更改HMS的 hive-site.xml 并重启HMS和HiveServer2：
如果想使用该特性(华为MRS除外)，需要更改HMS的 hive-site.xml 并重启HMS和HiveServer2：
```
<property>
@ -365,6 +365,33 @@ Doris 的权限管理功能提供了对 Catalog 层级的扩展，具体可参
```
华为的MRS需要更改hivemetastore-site.xml 并重启HMS和HiveServer2：
```
<property>
<name>metastore.transactional.event.listeners</name>
<value>org.apache.hive.hcatalog.listener.DbNotificationListener</value>
</property>
```
注意：value是在原有值的基础上以逗号分隔追加，而不是覆盖。例如MRS 3.1.0默认配置为
```
<property>
<name>metastore.transactional.event.listeners</name>
<value>com.huawei.bigdata.hive.listener.TableKeyFileManagerListener,org.apache.hadoop.hive.metastore.listener.FileAclListener</value>
</property>
```
我们需要改为
```
<property>
<name>metastore.transactional.event.listeners</name>
<value>com.huawei.bigdata.hive.listener.TableKeyFileManagerListener,org.apache.hadoop.hive.metastore.listener.FileAclListener,org.apache.hive.hcatalog.listener.DbNotificationListener</value>
</property>
```
> 使用建议： 无论是之前已经创建好的catalog现在想改为自动刷新，还是新创建的 catalog，都只需要把 `enable_hms_events_incremental_sync` 设置为true，重启fe节点，无需重启之前或之后再手动刷新元数据。
<version since="dev">