From 809bb4651895c3d951f4b98fc7366aa426f169c4 Mon Sep 17 00:00:00 2001 From: zhangdong <493738387@qq.com> Date: Wed, 7 Jun 2023 20:05:12 +0800 Subject: [PATCH] [doc](multi-catalog)huawei mrs event sync doc (#20466) --- .../lakehouse/multi-catalog/multi-catalog.md | 29 ++++++++++++++++++- .../lakehouse/multi-catalog/multi-catalog.md | 29 ++++++++++++++++++- 2 files changed, 56 insertions(+), 2 deletions(-) diff --git a/docs/en/docs/lakehouse/multi-catalog/multi-catalog.md b/docs/en/docs/lakehouse/multi-catalog/multi-catalog.md index d878de39ee..9f266f9fed 100644 --- a/docs/en/docs/lakehouse/multi-catalog/multi-catalog.md +++ b/docs/en/docs/lakehouse/multi-catalog/multi-catalog.md @@ -347,7 +347,7 @@ The automatic update feature involves the following parameters in fe.conf: 2. `hms_events_polling_interval_ms`: This specifies the interval between two readings, which is set to 10000 by default. (Unit: millisecond) 3. `hms_events_batch_size_per_rpc`: This specifies the maximum number of events that are read at a time, which is set to 500 by default. -To enable automatic update, you need to modify the hive-site.xml of HMS and then restart HMS and HiveServer2: +To enable automatic update(Excluding Huawei MRS), you need to modify the hive-site.xml of HMS and then restart HMS and HiveServer2: ``` @@ -365,6 +365,33 @@ To enable automatic update, you need to modify the hive-site.xml of HMS and then ``` +Huawei's MRS needs to change hivemetastore-site.xml and restart HMS and HiveServer2: + +``` + + metastore.transactional.event.listeners + org.apache.hive.hcatalog.listener.DbNotificationListener + +``` + +Note: Value is appended with commas separated from the original value, not overwritten.For example, the default configuration for MRS 3.1.0 is + +``` + + metastore.transactional.event.listeners + com.huawei.bigdata.hive.listener.TableKeyFileManagerListener,org.apache.hadoop.hive.metastore.listener.FileAclListener + +``` + +We need to change to + +``` + + metastore.transactional.event.listeners + com.huawei.bigdata.hive.listener.TableKeyFileManagerListener,org.apache.hadoop.hive.metastore.listener.FileAclListener,org.apache.hive.hcatalog.listener.DbNotificationListener + +``` + > Note: To enable automatic update, whether for existing Catalogs or newly created Catalogs, all you need is to set `enable_hms_events_incremental_sync` to `true`, and then restart the FE node. You don't need to manually update the metadata before or after the restart. diff --git a/docs/zh-CN/docs/lakehouse/multi-catalog/multi-catalog.md b/docs/zh-CN/docs/lakehouse/multi-catalog/multi-catalog.md index dd8dfd70b9..6a528d2302 100644 --- a/docs/zh-CN/docs/lakehouse/multi-catalog/multi-catalog.md +++ b/docs/zh-CN/docs/lakehouse/multi-catalog/multi-catalog.md @@ -347,7 +347,7 @@ Doris 的权限管理功能提供了对 Catalog 层级的扩展，具体可参 2. `hms_events_polling_interval_ms`: 读取 event 的间隔时间，默认值为 10000，单位：毫秒。 3. `hms_events_batch_size_per_rpc`: 每次读取 event 的最大数量，默认值为 500。 -如果想使用该特性，需要更改HMS的 hive-site.xml 并重启HMS和HiveServer2： +如果想使用该特性(华为MRS除外)，需要更改HMS的 hive-site.xml 并重启HMS和HiveServer2： ``` @@ -365,6 +365,33 @@ Doris 的权限管理功能提供了对 Catalog 层级的扩展，具体可参 ``` +华为的MRS需要更改hivemetastore-site.xml 并重启HMS和HiveServer2： + +``` + + metastore.transactional.event.listeners + org.apache.hive.hcatalog.listener.DbNotificationListener + +``` + +注意：value是在原有值的基础上以逗号分隔追加，而不是覆盖。例如MRS 3.1.0默认配置为 + +``` + + metastore.transactional.event.listeners + com.huawei.bigdata.hive.listener.TableKeyFileManagerListener,org.apache.hadoop.hive.metastore.listener.FileAclListener + +``` + +我们需要改为 + +``` + + metastore.transactional.event.listeners + com.huawei.bigdata.hive.listener.TableKeyFileManagerListener,org.apache.hadoop.hive.metastore.listener.FileAclListener,org.apache.hive.hcatalog.listener.DbNotificationListener + +``` + > 使用建议： 无论是之前已经创建好的catalog现在想改为自动刷新，还是新创建的 catalog，都只需要把 `enable_hms_events_incremental_sync` 设置为true，重启fe节点，无需重启之前或之后再手动刷新元数据。