# 如何使用 OBD 自动化部署多节点集群

## 部署规划

这一节介绍 OceanBase 集群三节点部署方法，需要通过中控机直接远程登录到 OceanBase 节点上部署启动 `observer` 和 `obproxy` 进程。

+ 机器信息如下：

| 机器类型 | 云主机 ECS                       |
|------|-------------------------------|
| IP   | 172.20.249.50 172.20.249.52 172.20.249.49 172.20.249.51                |
| 网卡名  | eth0                          |
| OS   | CentOS Linux release 8.4.2105 |
| CPU  | 4C                            |
| 内存   | 总内存 14G，可用内存 11G              |
| 磁盘1  | 云盘 /dev/vda  100G             |
| 磁盘2  | 云盘 /dev/vdb 100G              |

+ 机器划分如下：

| 角色       | 机器            | 备注                  |
|----------|---------------|---------------------|
| OBD      | 172.20.249.50 | 中控机，自动化部署软件         |
| OBSERVER | 172.20.249.52 | OceanBase 数据库 zone1 |
|          | 172.20.249.49 | OceanBase 数据库 zone2 |
|          | 172.20.249.51 | OceanBase 数据库 zone3 |
| OBPROXY  | 172.20.249.52 | OceanBase 访问反向代理    |
|          | 172.20.249.49 | OceanBase 访问反向代理    |
|          | 172.20.249.51 | OceanBase 访问反向代理    |
| OBCLIENT | 172.20.249.50 | OceanBase 命令行客户端    |

+ 磁盘划分
磁盘划分，这里就使用 LVM 技术对 `/dev/vdb` 进行划分。需要登录到每个节点上手动初始化。

```bash
# lvm 分盘
pvcreate /dev/vdb
vgcreate obvg /dev/vdb
lvcreate obvg -L 20G^C
lvcreate -L 20G obvg -n lvredo
lvcreate -l 100%FREE obvg -n lvdata

# 格式化文件系统
mkfs.ext4 /dev/obvg/lvdata
mkfs.ext4 /dev/obvg/lvredo

# 修改 mount 参数文件
vim /etc/fstab 
/dev/obvg/lvredo          /redo              ext4            defaults,noatime,nodiratime,nodelalloc,barrier=0        0 0
/dev/obvg/lvdata             /data         ext4            defaults,noatime,nodiratime,nodelalloc,barrier=0        0 0

# 挂载文件系统
mkdir -p /data /redo
vim /etc/fstab
mount /data
mount /redo
chown -R admin.admin /data /redo

# 检查
df -h

输出：
文件系统                 容量  已用  可用 已用% 挂载点
/dev/mapper/obvg-lvdata   59G   53M   56G    1% /data
/dev/mapper/obvg-lvredo   20G   45M   19G    1% /redo

  ```

## 编辑 OBD 配置文件

OBD 针对不同的部署场景提供不同的配置文件。这些配置文件示例在 OceanBase 开源项目地址里，具体是：[https://github.com/oceanbase/obdeploy/tree/master/example](https://github.com/oceanbase/obdeploy/tree/master/example) 。

如果是部署单节点版本，就下载其中两个配置文件：

+ 部署三节点 `observer` 进程： [https://github.com/oceanbase/obdeploy/blob/master/example/mini-distributed-example.yaml](https://github.com/oceanbase/obdeploy/blob/master/example/mini-distributed-example.yaml)
+ 部署三节点 `observer` 和 `obproxy` 进程：[https://github.com/oceanbase/obdeploy/blob/master/example/mini-distributed-with-obproxy-example.yaml](https://github.com/oceanbase/obdeploy/blob/master/example/mini-distributed-with-obproxy-example.yaml)

这里仿照生产环境，选择第二种部署配置文件。

```bash
[admin@obce00 ~]$ cat obce-3zones.yaml
# Only need to configure when remote login is required
user:
   username: admin
#   password: your password if need
   key_file: /home/admin/.ssh/id_rsa.pub
   port: your ssh port, default 22
#   timeout: ssh connection timeout (second), default 30
oceanbase-ce:
  servers:
    - name: obce01
      # Please don't use hostname, only IP can be supported
      ip: 172.20.249.52
    - name: obce02
      ip: 172.20.249.49
    - name: obce03
      ip: 172.20.249.51
  global:
    # Please set devname as the network adaptor's name whose ip is  in the setting of severs.
    # if set severs as "127.0.0.1", please set devname as "lo"
    # if current ip is 192.168.1.10, and the ip's network adaptor's name is "eth0", please use "eth0"
    devname: eth0
    cluster_id: 2
    # please set memory limit to a suitable value which is matching resource.
    memory_limit: 8G # The maximum running memory for an observer
    system_memory: 3G # The reserved system memory. system_memory is reserved for general tenants. The default value is 30G.
    stack_size: 512K
    cpu_count: 16
    cache_wash_threshold: 1G
    __min_full_resource_pool_memory: 268435456
    workers_per_cpu_quota: 10
    schema_history_expire_time: 1d
    # The value of net_thread_count had better be same as cpu's core number.
    net_thread_count: 4
    major_freeze_duty_time: Disable
    minor_freeze_times: 10
    enable_separate_sys_clog: 0
    enable_merge_by_turn: FALSE
    #datafile_disk_percentage: 20 # The percentage of the data_dir space to the total disk space. This value takes effect only when datafile_size is 0. The default value is 90.
    datafile_size: 50G
    syslog_level: WARN # System log level. The default value is INFO.
    enable_syslog_wf: false # Print system logs whose levels are higher than WARNING to a separate log file. The default value is true.
    enable_syslog_recycle: true # Enable auto system log recycling or not. The default value is false.
    max_syslog_file_count: 10 # The maximum number of reserved log files before enabling auto recycling. The default value is 0.
    # observer cluster name, consistent with obproxy's cluster_name
    appname: obce-3zones
    root_password: 0EI5N08d # root user password, can be empty
    proxyro_password: uY7Yf8zx # proxyro user pasword, consistent with obproxy's observer_sys_password, can be empty
  obce01:
    mysql_port: 2881 # External port for OceanBase Database. The default value is 2881.
    rpc_port: 2882 # Internal port for OceanBase Database. The default value is 2882.
    #  The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field.
    home_path: /home/admin/oceanbase-ce
    # The directory for data storage. The default value is $home_path/store.
    data_dir: /data
    # The directory for clog, ilog, and slog. The default value is the same as the data_dir value.
    redo_dir: /redo
    zone: zone1
  obce02:
    mysql_port: 2881 # External port for OceanBase Database. The default value is 2881.
    rpc_port: 2882 # Internal port for OceanBase Database. The default value is 2882.
    #  The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field.
    home_path: /home/admin/oceanbase-ce
    # The directory for data storage. The default value is $home_path/store.
    data_dir: /data
    # The directory for clog, ilog, and slog. The default value is the same as the data_dir value.
    redo_dir: /redo
    zone: zone2
  obce03:
    mysql_port: 2881 # External port for OceanBase Database. The default value is 2881.
    rpc_port: 2882 # Internal port for OceanBase Database. The default value is 2882.
    #  The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field.
    home_path: /home/admin/oceanbase-ce
    # The directory for data storage. The default value is $home_path/store.
    data_dir: /data
    # The directory for clog, ilog, and slog. The default value is the same as the data_dir value.
    redo_dir: /redo
    zone: zone3
obproxy:
  servers:
    - 172.20.249.52
    - 172.20.249.49
    - 172.20.249.51
  # Set dependent components for the component.
  # When the associated configurations are not done, OBD will automatically get the these configurations from the dependent components.
  depends:
    - oceanbase-ce
  global:
    listen_port: 2883 # External port. The default value is 2883.
    prometheus_listen_port: 2884 # The Prometheus port. The default value is 2884.
    home_path: /home/admin/obproxy
    # oceanbase root server list
    # format: ip:mysql_port;ip:mysql_port
    rs_list: 172.20.249.52:2881;172.20.249.49:2881;172.20.249.51:2881
    enable_cluster_checkout: false
    # observer cluster name, consistent with oceanbase-ce's appname
    cluster_name: obce-3zones
    obproxy_sys_password: 0MdTv1tm # obproxy sys user password, can be empty
    observer_sys_password: uY7Yf8zx # proxyro user pasword, consistent with oceanbase-ce's proxyro_password, can be empty
    
```

这个配置文件是专门针对最小内存（可用内存大于 8G）的节点配置，里面指定了很多进程 `observer` 的启动参数。注意 `yaml` 的格式，每个配置项后面冒号（`:`) 跟后面的值之间必须有个空格（`' '`)。
下面就关键的几个参数补充说明如下：

| 配置类          | 配置项名                                     | 配置值                                                      | 备注                                                                                                                                            |
|--------------|------------------------------------------|----------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------|
| user         | username                                 | admin                                                    | 中控机连接OceanBase 节点的用户名，也是 OceanBase 要部署的用户名。                                                                                                   |
|              | key_file                                 | /home/admin/.ssh/id_rsa.pub                              | 中控机上 SSH用的RSA 公钥。                                                                                                                             |
|              | port                                     | 22                                                       | OceanBase 集群节点的 SSH 端口，默认是22。如果不是就修改这里。                                                                                                       |
| oceanbase-ce | servers                                  | 指定所有机器列表                                            | 每个机器是用 `- name 机器标识名 (换行)ip: 机器ip` 指定。多个机器就指定多次，以后会优化    |
|              | home_path                                | /home/admin/oceanbase-ce                                 | 指定到普通用户（admin）的目录下，为区别于企业版，文件名叫 `oceanbase-ce` 。                                                                                              |
|              | data_dir                                 | /data                                                    | 指向独立的磁盘，这里使用前面分配的 LV（`lvdata`)。实际存储 OceanBase 的数据文件（`block_file`)。                                                                                   |
|              | redo_dir                                 | /redo                                                    | 指向独立的磁盘，这里使用前面分配的LV（`lvredo`)。实际存储 OceanBase 的事务日志、`sstable` 日志等。                                                                                    |
|              | devname                                  | eth0                                                     | 这个是跟 servers 里指定的 IP 对应的网卡。如果前面 IP 是 127.0.0.1 ，那么这里就填  lo 。通过 ip addr 命令可以查看 IP 和网卡对应关系。                                                     |
|              | mysql_port                               | 2881                                                     | 进程 `observer` 的连接端口，默认是 2881 。后面 OceanBase 客户端直连这个端口可以访问该节点。                                                                                         |
|              | rpc_port                                 | 2882                                                     | 进程 `observer` 跟其他节点进程之间的 RPC 通信端口，默认是 2882 。                                                                                                  |
|              | zone                                     | zone1 zone2  zone3                                       | `zone` 是逻辑机房的概念。三副本集群下有三个 `zone` 。                                                                                                            |
|              | cluster_id                               | 2                                                        | OceanBase 集群ID 标识，不同集群不要重复即可。                                                                                                                 |
|              | memory_limit                             | 8G                                                       | 进程 `observer` 能从OS 获取的最大内存，最小不少于 8G 。如果机器内存丰富的话，这个参数可以大一些。                                                                                    |
|              | system_memory                            | 4G                                                       | 进程 `observer` 留给集群内部用的保留内存，这个会占用上面 `memory_limit` 的内存，留给业务租户的就更少。                                                                             |
|              | datafile_size | datafile_disk_percentage |                                                          | 这两个参数 2 选 1。用来指定该节点数据文件（`block_file`）大小的。可以按大小指定，或者按磁盘空间的百分比指定。这里示例磁盘空间很小，为精确控制，指定数据文件大小。当数据文件和事务日志文件是共用一个磁盘的时候，则必须指定数据文件具体大小，以避免日志文件的目录空间不够。 |
|              | syslog_level                             | WARN 或 ERROR                                             | 运行日志的日志级别，有 INFO | WARN | ERROR 等几个级别。级别越低，日志量越大。进程 `observer` 的日志量非常大，如果磁盘空间不大的话，就调整为 WARN 或 ERROR 吧。                                        |
|              | enable_syslog_recycle                    | TRUE                                                     | 指定运行日志是否以滚动方式输出，最多保留 指定数量的运行日志。                                                                                                               |
|              | max_syslog_file_count                    | 10                                                       | 根据磁盘空间大小定，这里默认保留最多 10 个历史运行日志文件。                                                                                                              |
|              | root_password                            | 随机字符串                                                    | OceanBase 集群的超级管理员 `root@sys` 的密码，默认是空，建议设置复杂的密码。                                                                                                    |
|              | proxyro_password                         | 随机字符串                                                    | OBPROXY 连接 OB集群使用的账户名（proxyro) 的密码                                                                                                            |
| obproxy      | servers                                  | 任意机器IP                                                   | OBPROXY 可以部署在应用服务器、中控机或者 OceanBase 机器上。这里选择 OceanBase 机器。                                                                                                   |
|   | depends | 依赖的配置节|   通常指定依赖的集群配置，会自动复用集群的 `proxyro` 密码、集群名 `cluster_name`、`rs_list` 等等。|
|              | listen_port                              | 2883                                                     | OBPROXY 监听端口，默认 2883 。                                                                                                                        |
|              | prometheus_listen_port                   | 2884                                                     | prometheus 监听端口，默认 2884。                                                                                                                      |
|              | home_path                                | /home/admin/obproxy                                      | OBPROXY 默认安装路径，建议在普通用户 admin 下。                                                                                                               |
|              | rs_list                                  | 172.20.249.52:2881;172.20.249.49:2881;172.20.249.51:2881 | OceanBase 集群 rootservice 服务地址，由 sys 租户的三副本所在节点IP 组成。 可以手动指定，也可以不指定，依赖前面的 `depends` 配置节自动从 OceanBase 集群配置里获取。                                                                                                |
|              | enable_cluster_checkout                  | FALSE                                                    |                                                                                                                                               |
|              | cluster_name                             | obce-3zones                                              | OceanBase 集群名字                                                                                                                                       |
|              | obproxy_sys_password                     | 随机字符串                                                    | OBPROXY 管理员账户（`proxysys`）的密码。                                                                                                                 |
|              | observer_sys_password                    | 跟 proxyro_password 一致                                    | OBPROXY 连接 OB集群使用的账户名（proxyro) 的密码                                                                                                            |

当上面部署成功后，OBD 会把配置文件 `obce-3zones.yaml` 复制到自己的工作目录里（`~/.obd/cluster/obce-3zones/config.yaml` ），后期再改外面这个 `obce-3zones.yaml` 文件，是不生效的。

## OBD 部署集群

配置文件准备好后，就可以部署这个配置文件对应的集群了，部署内容主要包含：

+ 复制软件到相应节点，并安装软件。
+ 在相应节点创建相关目录。

部署使用命令：`obd cluster deploy [集群名] -c 集群配置文件`

```bash
[admin@obce00 ~]$ obd cluster deploy obce-3zones -c obce-3zones.yaml
oceanbase-ce-3.1.0 already installed.
obproxy-3.1.0 already installed.
+-----------------------------------------------------------------------------+
|                                   Packages                                  |
+--------------+---------+---------+------------------------------------------+
| Repository   | Version | Release | Md5                                      |
+--------------+---------+---------+------------------------------------------+
| oceanbase-ce | 3.1.0   | 3.el8   | 84bd2fe27f8b8243cc57d8a3f68b4c50f94aab80 |
| obproxy      | 3.1.0   | 1.el8   | d242ea5fe45222b8f61c3135ba2aaa778c61ea22 |
+--------------+---------+---------+------------------------------------------+
Repository integrity check ok
Parameter check ok
Open ssh connection ok
Remote oceanbase-ce-3.1.0-84bd2fe27f8b8243cc57d8a3f68b4c50f94aab80 repository install ok
Remote oceanbase-ce-3.1.0-84bd2fe27f8b8243cc57d8a3f68b4c50f94aab80 repository lib check ok
Remote obproxy-3.1.0-d242ea5fe45222b8f61c3135ba2aaa778c61ea22 repository install ok
Remote obproxy-3.1.0-d242ea5fe45222b8f61c3135ba2aaa778c61ea22 repository lib check ok
Cluster status check ok
Initializes cluster work home ok
Initializes cluster work home ok
obce-3zones deployed

```

检查集群部署状态。

```bash
[admin@obce00 ~]$ obd cluster list
+----------------------------------------------------------------------+
|                             Cluster List                             |
+-------------+--------------------------------------+-----------------+
| Name        | Configuration Path                   | Status (Cached) |
+-------------+--------------------------------------+-----------------+
| obce-3zones | /home/admin/.obd/cluster/obce-3zones | deployed        |
+-------------+--------------------------------------+-----------------+
```

## OBD 启动和初始化集群

上面 `deploy` 操作只是安装了软件和准备初始化目录，还需要启动集群节点并初始化集群，使用 `obd cluster start` 命令。

```bash
obd cluster start obce-3zones

输出：
[admin@obce00 ~]$ obd cluster start obce-3zones
Get local repositories and plugins ok
Open ssh connection ok
Cluster param config check ok
Check before start observer ok
[WARN] (172.20.249.52) The recommended value of fs.aio-max-nr is 1048576 (Current value: 65536)
[WARN] (172.20.249.52) The recommended number of open files is 655350 (Current value: 65535)
[WARN] (172.20.249.49) The recommended value of fs.aio-max-nr is 1048576 (Current value: 65536)
[WARN] (172.20.249.49) The recommended number of open files is 655350 (Current value: 65535)
[WARN] (172.20.249.51) The recommended value of fs.aio-max-nr is 1048576 (Current value: 65536)
[WARN] (172.20.249.51) The recommended number of open files is 655350 (Current value: 65535)

Check before start obproxy ok
Start observer ok
observer program health check ok
Connect to observer ok
Initialize cluster
Cluster bootstrap ok
Wait for observer init ok
+-------------------------------------------------+
|                     observer                    |
+---------------+---------+------+-------+--------+
| ip            | version | port | zone  | status |
+---------------+---------+------+-------+--------+
| 172.20.249.49 | 3.1.0   | 2881 | zone2 | active |
| 172.20.249.51 | 3.1.0   | 2881 | zone3 | active |
| 172.20.249.52 | 3.1.0   | 2881 | zone1 | active |
+---------------+---------+------+-------+--------+

Start obproxy ok
obproxy program health check ok
Connect to obproxy ok
Initialize cluster
+-------------------------------------------------+
|                     obproxy                     |
+---------------+------+-----------------+--------+
| ip            | port | prometheus_port | status |
+---------------+------+-----------------+--------+
| 172.20.249.52 | 2883 | 2884            | active |
| 172.20.249.49 | 2883 | 2884            | active |
| 172.20.249.51 | 2883 | 2884            | active |
+---------------+------+-----------------+--------+
obce-3zones running

```

如果集群节点内核参数和会话限制参数不符合要求，安装会给出提示。
这个命令会在 `bootstrap` 要几分钟。当可用内存不足 8G 或者 日志目录剩余可用空间比例不足 5% 的时候，这个 `bootstrap` 是很可能会失败的。

接下来进一步确认集群初始化成功。这个步骤是可选的。第一次学习或生产部署的时候，建议检查一下。

+ 首先查看启动后的集群状态。

```bash

[admin@obce00 ~]$ obd cluster list
+----------------------------------------------------------------------+
|                             Cluster List                             |
+-------------+--------------------------------------+-----------------+
| Name        | Configuration Path                   | Status (Cached) |
+-------------+--------------------------------------+-----------------+
| obce-3zones | /home/admin/.obd/cluster/obce-3zones | running         |
+-------------+--------------------------------------+-----------------+

```

+ 检查 OceanBase 集群各个节点进程信息。

OceanBase 是单进程软件，进程名叫 `observer` ，可以用下面命令查看这个进程。

```bash
IPS="172.20.249.52 172.20.249.49 172.20.249.51"
for ob in $IPS;do echo $ob; ssh $ob "ps -ef | grep observer | grep -v grep "; done

输出：
[admin@obce00 oceanbase-ce]$ for ob in $IPS;do echo $ob; ssh $ob "ps -ef | grep observer | grep -v grep "; done
172.20.249.52
admin       6987       1 69 08:35 ?        01:38:26 /home/admin/oceanbase-ce/bin/observer -r 172.20.249.52:2882:2881;172.20.249.49:2882:2881;172.20.249.51:2882:2881 -o __min_full_resource_pool_memory=268435456,memory_limit=8G,system_memory=3G,stack_size=512K,cpu_count=16,cache_wash_threshold=1G,workers_per_cpu_quota=10,schema_history_expire_time=1d,net_thread_count=4,major_freeze_duty_time=Disable,minor_freeze_times=10,enable_separate_sys_clog=0,enable_merge_by_turn=False,datafile_size=50G,enable_syslog_wf=False,enable_syslog_recycle=True,max_syslog_file_count=10,root_password=0EI5N08d,redo_dir=/redo -z zone1 -p 2881 -P 2882 -n obce-3zones -c 2 -d /data -i eth0 -l WARN
172.20.249.49
admin       7064       1 87 08:35 ?        02:02:59 /home/admin/oceanbase-ce/bin/observer -r 172.20.249.52:2882:2881;172.20.249.49:2882:2881;172.20.249.51:2882:2881 -o __min_full_resource_pool_memory=268435456,memory_limit=8G,system_memory=3G,stack_size=512K,cpu_count=16,cache_wash_threshold=1G,workers_per_cpu_quota=10,schema_history_expire_time=1d,net_thread_count=4,major_freeze_duty_time=Disable,minor_freeze_times=10,enable_separate_sys_clog=0,enable_merge_by_turn=False,datafile_size=50G,enable_syslog_wf=False,enable_syslog_recycle=True,max_syslog_file_count=10,root_password=0EI5N08d,redo_dir=/redo -z zone2 -p 2881 -P 2882 -n obce-3zones -c 2 -d /data -i eth0 -l WARN
172.20.249.51
admin       6920       1 72 08:35 ?        01:42:42 /home/admin/oceanbase-ce/bin/observer -r 172.20.249.52:2882:2881;172.20.249.49:2882:2881;172.20.249.51:2882:2881 -o __min_full_resource_pool_memory=268435456,memory_limit=8G,system_memory=3G,stack_size=512K,cpu_count=16,cache_wash_threshold=1G,workers_per_cpu_quota=10,schema_history_expire_time=1d,net_thread_count=4,major_freeze_duty_time=Disable,minor_freeze_times=10,enable_separate_sys_clog=0,enable_merge_by_turn=False,datafile_size=50G,enable_syslog_wf=False,enable_syslog_recycle=True,max_syslog_file_count=10,root_password=0EI5N08d,redo_dir=/redo -z zone3 -p 2881 -P 2882 -n obce-3zones -c 2 -d /data -i eth0 -l WARN
```

从进程里看，可执行文件是 `/home/admin/oceanbase-ce/bin/observer` ，实际上它是个软链接。

```bash
[admin@obce00 oceanbase-ce]$ ll /home/admin/oceanbase-ce/bin/observer
lrwxrwxrwx 1 admin admin 100 Sep 11 17:16 /home/admin/oceanbase-ce/bin/observer -> /home/admin/.obd/repository/oceanbase-ce/3.1.0/84bd2fe27f8b8243cc57d8a3f68b4c50f94aab80/bin/observer
```

进程启动的时候，通过 `-o` 指定了很多参数，这些参数都是在前面 OBD 集群部署配置文件里指定的。

+ 检查 OceanBase 集群各个节点监听状况。

```bash
IPS="172.20.249.52 172.20.249.49 172.20.249.51"
for ob in $IPS;do echo $ob; ssh $ob "netstat -ntlp"; done

输出：
[admin@obce00 ~]$ for ob in $IPS;do echo $ob; ssh $ob "netstat -ntlp"; done
172.20.249.52
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:2881            0.0.0.0:*               LISTEN      6987/observer
tcp        0      0 0.0.0.0:2882            0.0.0.0:*               LISTEN      6987/observer
tcp        0      0 0.0.0.0:2883            0.0.0.0:*               LISTEN      7640/obproxy
tcp        0      0 0.0.0.0:2884            0.0.0.0:*               LISTEN      7640/obproxy
172.20.249.49
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:2881            0.0.0.0:*               LISTEN      7064/observer
tcp        0      0 0.0.0.0:2882            0.0.0.0:*               LISTEN      7064/observer
tcp        0      0 0.0.0.0:2883            0.0.0.0:*               LISTEN      7718/obproxy
tcp        0      0 0.0.0.0:2884            0.0.0.0:*               LISTEN      7718/obproxy
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -
172.20.249.51
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:2881            0.0.0.0:*               LISTEN      6920/observer
tcp        0      0 0.0.0.0:2882            0.0.0.0:*               LISTEN      6920/observer
tcp        0      0 0.0.0.0:2883            0.0.0.0:*               LISTEN      7574/obproxy
tcp        0      0 0.0.0.0:2884            0.0.0.0:*               LISTEN      7574/obproxy
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -
```

## 连接 OceanBase 集群的内部实例（`sys`)

传统的 mysql 客户端可以连接 OceanBase 社区版，前提是 mysql 的版本是 5.5/5.6/5.7 。OceanBase 也提供自己的客户端工具 `obclient` 需要安装使用。
跟传统MySQL 不一样的地方是 OBPROXY 的连接端口是 2883 ， 连接用户名是 ：`root@sys#集群名` ，密码是前面 OBD 配置文件里指定的。

```bash
mysql -h 172.20.249.52 -uroot@sys#obce-3zones -P2883 -p0EI5N08d -c -A oceanbase

输出：
[admin@obce00 ~]$ mysql -h 172.20.249.52 -uroot@sys#obce-3zones -P2883 -p0EI5N08d -c -A oceanbase
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.6.25 OceanBase 3.1.0 (r3-b20901e8c84d3ea774beeaca963c67d7802e4b4e) (Built Aug 10 2021 08:10:38)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [oceanbase]> show databases;
+--------------------+
| Database           |
+--------------------+
| oceanbase          |
| information_schema |
| mysql              |
| SYS                |
| LBACSYS            |
| ORAAUDITOR         |
| test               |
+--------------------+
7 rows in set (0.002 sec)

MySQL [oceanbase]> select a.zone,concat(a.svr_ip,':',a.svr_port) observer, cpu_total, (cpu_total-cpu_assigned) cpu_free, round(mem_total/1024/1024/1024) mem_total_gb, round((mem_total-mem_assigned)/1024/1024/1024) mem_free_gb, usec_to_time(b.last_offline_time) last_offline_time, usec_to_time(b.start_service_time) start_service_time, b.status, usec_to_time(b.stop_time) stop_time, b.build_version 
from __all_virtual_server_stat a join __all_server b on (a.svr_ip=b.svr_ip and a.svr_port=b.svr_port)
order by a.zone, a.svr_ip
;

+-------+--------------------+-----------+----------+--------------+-------------+----------------------------+----------------------------+--------+----------------------------+------------------------------------------------------------------------+
| zone  | observer           | cpu_total | cpu_free | mem_total_gb | mem_free_gb | last_offline_time          | start_service_time         | status | stop_time                  | build_version                                                          |
+-------+--------------------+-----------+----------+--------------+-------------+----------------------------+----------------------------+--------+----------------------------+------------------------------------------------------------------------+
| zone1 | 172.20.249.52:2882 |        14 |     11.5 |            5 |           4 | 1970-01-01 08:00:00.000000 | 2021-09-12 08:36:06.357140 | active | 1970-01-01 08:00:00.000000 | 3.1.0_3-b20901e8c84d3ea774beeaca963c67d7802e4b4e(Aug 10 2021 08:10:38) |
| zone2 | 172.20.249.49:2882 |        14 |     11.5 |            5 |           4 | 1970-01-01 08:00:00.000000 | 2021-09-12 08:36:07.605244 | active | 1970-01-01 08:00:00.000000 | 3.1.0_3-b20901e8c84d3ea774beeaca963c67d7802e4b4e(Aug 10 2021 08:10:38) |
| zone3 | 172.20.249.51:2882 |        14 |     11.5 |            5 |           4 | 1970-01-01 08:00:00.000000 | 2021-09-12 08:36:07.631981 | active | 1970-01-01 08:00:00.000000 | 3.1.0_3-b20901e8c84d3ea774beeaca963c67d7802e4b4e(Aug 10 2021 08:10:38) |
+-------+--------------------+-----------+----------+--------------+-------------+----------------------------+----------------------------+--------+----------------------------+------------------------------------------------------------------------+
3 rows in set (0.004 sec)
```

在数据库列表里看到 `oceanbase` 这个数据库，就表示集群初始化成功。

`obclient` 安装和使用示例。

```bash
sudo rpm -ivh /tmp/obd/obclient-2.0.0-2.el8.x86_64.rpm /tmp/obd/libobclient-2.0.0-2.el8.x86_64.rpm

obclient -h 172.20.249.52 -uroot@sys#obce-3zones -P2883 -p0EI5N08d -c -A oceanbase

```
