diff --git a/README.md b/README.md index 57b1b16580..709f323252 100644 --- a/README.md +++ b/README.md @@ -183,11 +183,6 @@ Doris provides support for Spark/Flink to read data stored in Doris through Conn 🔗[apache/doris-spark-connector](https://github.com/apache/doris-spark-connector) -### 🛠 Doris Manager - -Doris provides one-click visual automatic installation and deployment, cluster management and monitoring tools for clusters. - -🔗[apache/doris-manager](https://github.com/apache/doris-manager) ## 🌈 Community and Support diff --git a/docs/en/docs/ecosystem/doris-manager/cluster-managenent.md b/docs/en/docs/ecosystem/doris-manager/cluster-managenent.md deleted file mode 100644 index 138061d0dd..0000000000 --- a/docs/en/docs/ecosystem/doris-manager/cluster-managenent.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -{ 'title': 'Cluster management', 'language': 'en' } ---- - - - -# Cluster management - -The super administrator and space administrator can mainly perform the following operations under the cluster module: - -- View cluster overview -- View node list -- Edit parameter configuration - -## Cluster overview - -### View basic cluster information - -Cluster function, showing a cluster-based monitoring panel. - -On the home page, click "Cluster" in the navigation bar to enter the cluster function. - -![](/images/doris-manager/clustermanagenent-1.png) - -The operation and maintenance monitoring panel provides various performance monitoring indicators of the cluster for users to gain insight into the cluster status. Users can control the start and stop operations of the cluster through buttons in the upper right corner. - -### View cluster resource usage - -Users can view disk usage through pie charts, and view the number of databases, etc. - -## Node list - -Displays information about FE nodes, BE nodes, and brokers in the cluster. -Provides fields including Node ID, Node Type, Host IP, and Node Status. - -![](/images/doris-manager/clustermanagenent-2.png) - -## Parameter configuration - -Parameter configuration provides parameter name, parameter type, parameter value type, thermal effect and operation fields. - -![](/images/doris-manager/clustermanagenent-3.png) - -- **Operation**: Click the "Edit" button, you can edit and modify the corresponding configuration value, you can choose the corresponding effective method; click the "View current value" button, you can view the current value corresponding to the host IP - -![](/images/doris-manager/clustermanagenent-4.png) - -![](/images/doris-manager/clustermanagenent-5.png) diff --git a/docs/en/docs/ecosystem/doris-manager/compiling-deploying.md b/docs/en/docs/ecosystem/doris-manager/compiling-deploying.md deleted file mode 100644 index a6f242d8f4..0000000000 --- a/docs/en/docs/ecosystem/doris-manager/compiling-deploying.md +++ /dev/null @@ -1,131 +0,0 @@ ---- -{ - "title": "Compile and deploy", - "language": "en" -} ---- - - - -# Compile and deploy - -## Compile - -The build.sh script directly under the manager path -```shell -cd incubator-doris-manager -sh build.sh -```` -After the compilation is completed, the output directory of the installation package will be generated under the manager path. The directory structure is as follows -````text -├── agent //agent directory -│ ├── bin -│ │ ├── agent_start.sh -│ │ ├── agent_stop.sh -│ │ └── download_doris.sh -│ ├── config -│ │ └── application.properties -│ └── lib -│ └── dm-agent.jar -└── server //server directory - ├── bin - │ ├── start_manager.sh //Doris Manager startup script - │ └── stop_manager.sh //Doris Manager stop script - ├── conf - │ └── manager.conf //Doris Manager configuration file - ├── lib - │ └── doris-manager.jar //Doris Manager's running package doris-manager.jar - └── web-resource -```` -## Run - -### 1 Configuration - -Modify the configuration file `server/conf/manager.conf`, and focus on the following configuration items: -````$xslt -The service's startup http port -STUDIO_PORT=8080 - -The type of database where the backend data is stored, including mysql/h2/postgresql. The default is to support mysql -MB_DB_TYPE=mysql - -Database connection information -If it is a configured h2 type database, you do not need to configure this information, and the data will be stored locally as a local file -h2 data file storage path, directly stored in the current path by default -H2_FILE_PATH= - -If it is mysql/postgresql, you need to configure the following connection information -database address -MB_DB_HOST= - -database port -MB_DB_PORT=3306 - -database access port -MB_DB_USER= - -Database access password -MB_DB_PASS= - -database name of the database -MB_DB_DBNAME= - -The path where the service runs, which is directly stored in the log folder of the current running path by default. -LOG_PATH= - -The length of the waiting queue of the web container, the default is 100. The queue is also used as a buffer pool, but it cannot be infinitely long. It not only consumes memory, but also consumes CPU when entering the queue. -WEB_ACCEPT_COUNT=100 - -The maximum number of worker threads for the web container, 200 by default. (usually the number of CPU cores * 200) -WEB_MAX_THREADS=200 - -The minimum number of working idle threads for the web container, the default is 10. (Appropriately increase some to cope with the sudden increase in traffic) -WEB_MIN_SPARE_THREADS=10 - -The maximum number of connections for the web container, the default is 10000. (Appropriately increase some to cope with the sudden increase in traffic) -WEB_MAX_CONNECTIONS=10000 - -The maximum number of connections to access the database connection pool, the default is 10 -DB_MAX_POOL_SIZE=20 - -The minimum number of idle connections to access the database connection pool, the default is 10 -DB_MIN_IDLE=10 -```` - -### 2 Start - -After the configuration modification is completed, start doris manger -````shell -cd server -sh bin/start_manager.sh -```` - -Check the logs in the logs to determine whether the program started successfully - -### 3 Use - -Doris Manager presets a super administrator user with the following information: - -````$xslt -Username: Admin -Password: Admin@123 -```` - -To ensure safe use, please change your password after logging in! diff --git a/docs/en/docs/ecosystem/doris-manager/initializing.md b/docs/en/docs/ecosystem/doris-manager/initializing.md deleted file mode 100644 index 3c867d81c6..0000000000 --- a/docs/en/docs/ecosystem/doris-manager/initializing.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -{ - "title": "Initialize", - "language": "en" -} ---- - - - -# Initialize - -After the deployment is complete, the super administrator needs to complete the local initialization. - -## Manage users - -The first step of initialization is to manage users, which mainly completes the selection and configuration of authentication methods. Currently Doris Manger supports local user authentication. - -![](/images/doris-manager/initializing-1.png) - -### Local user authentication - -Local user authentication is the user system that comes with Doris Manger. User registration can be completed by filling in the user name, email address and password. User addition, information modification, deletion and permission relationship are all completed locally. - -![](/images/doris-manager/initializing-2.png) - -At this point, the local initialization process has been completed. Super administrators can create spaces, space administrators can enter the space, manage the space, add and invite users to enter the space for data analysis, etc. \ No newline at end of file diff --git a/docs/en/docs/ecosystem/doris-manager/space-list.md b/docs/en/docs/ecosystem/doris-manager/space-list.md deleted file mode 100644 index 65dd65028a..0000000000 --- a/docs/en/docs/ecosystem/doris-manager/space-list.md +++ /dev/null @@ -1,237 +0,0 @@ ---- -{ - "title": "Space list", - "language": "en" -} ---- - - - -# Space list - -The super administrator can perform the following operations in the space list: - -- Perform new cluster and cluster hosting operations - -- Recovery and deletion of unfinished spaces - -- Completed space deletion operation - -The space administrator can mainly perform the following operations in the space list: - -- View authorized space information - -## Completed space - -The super administrator can operate the completed space through the button to the right of the space name. Space administrators can click to enter the space to manage clusters or data in the space. - -![](/images/doris-manager/spacelist-1.png) - -## Unfinished space - -Doris Manger provides a draft save function of the space creation process to record the incomplete space creation process. Super administrators can view the list of unfinished spaces by switching tabs, and perform recovery or deletion operations. - -![](/images/doris-manager/spacelist-2.png) - -# New space - -There are two ways to create a new space: new cluster and cluster hosting. - -## New cluster - -### 1 Registration space - -Space information includes space name, space introduction, and selection of space administrators. - -Space name and administrator are required/optional fields. - -![](/images/doris-manager/spacelist-3.png) - -### 2 Add host - -![](/images/doris-manager/spacelist-4.png) - -#### Configure SSH login-free - -Doris Manager needs to distribute the Agent installation package during installation, so it is necessary to configure SSH login-free on the server (agent01) where Doris is to be installed. - -```shell -#1. To log in to the server, you need to use the manager and agent accounts to be consistent -su - xxx -pwd -#2. Generate a key pair on the machine where doris manager is deployed -ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa - -#3. Copy the public key to the machine agent01 -scp ~/.ssh/id_rsa.pub root@agent01:~ - -#4. Log in to agent01 and append the public key to authorized_keys -cat ~/id_rsa.pub >> .ssh/authorized_keys - -#5. After doing this, we can log in to agent01 without password on the doris manger machine -ssh agent01@xx.xxx.xx.xx -```` -For details, please refer to: https://blog.csdn.net/universe_hao/article/details/52296811 - -In addition, it should be noted that the permissions of the .ssh directory are 700, and the permissions of the authorized_keys and private keys under it are 600. Otherwise, you will not be able to log in without a password due to permission issues. We can see that the known_hosts file will be generated after logging in. At the same time, when starting doris, you need to use a password-free login account. - - -When adding nodes in the second step of the Doris Manager installation cluster, use the private key of the doris manager machine, that is, ~/.ssh/id_rsa (note: including the head and tail of the key file) - -#### Host list -Enter the host IP to add a new host, or add it in batches. - -### 3 Installation options - -#### Get the installation package - -1. Code package path - - When deploying a cluster through Doris Manager, you need to provide the compiled Doris installation package. You can compile it yourself from the Doris source code. - -`Doris Manager will pull the Doris installation package through http. If you need to build your own http service, please refer to the bottom of the document - Self-built http service`. - -#### Specify the installation path - -1. Doris and Doris Manger Agent will be installed in this directory. Make sure this directory is dedicated to Doirs and related components. -2. Specify the Agent startup port, the default is 8001, if there is a conflict, you can customize it. - -### 4 Verify the host - -The system will automatically perform verification according to the host status. When the verification is completed, the Agent will start sending back the heartbeat, and you can click to proceed to the next step. - -![](/images/doris-manager/spacelist-5.png) - -### 5 Planning Nodes - -Click the Assign Node button to plan FE/BE/Broker nodes for the host. - -![](/images/doris-manager/spacelist-6.png) - -### 6 Configuration Parameters - -Configure parameters for the nodes planned in the previous step. You can use the default values ​​or turn on the custom configuration switch to customize the configuration. - -### 7 Deploy the cluster - -The system will automatically perform verification according to the status of the host installation progress. When the verification is completed, it will start the node and return the heartbeat. You can click to proceed to the next step. - -![](/images/doris-manager/spacelist-7.png) - -### 8 Complete the creation - -Complete the above steps to complete the new cluster. - -![](/images/doris-manager/spacelist-8.png) - -## Cluster hosting - -### 1 Registration space - -Space information includes space name, space introduction, and selection of space administrators. - -Space name and administrator are required/optional fields. - -### 2 Connect to the cluster - -Cluster information includes cluster address, HTTP port, JDBC port, cluster username, and cluster password. Users can fill in according to their own cluster information. - -Click the Link Test button to test it. - -### 3 Hosting Options - -![](/images/doris-manager/spacelist-9.png) - -#### Configure SSH login-free - -Doris Manager needs to distribute the Agent installation package during installation, so it is necessary to configure SSH login-free on the server (agent01) where Doris is to be installed. - -```shell -#1. To log in to the server, you need to use the manger and agent accounts to be consistent -su - xxx -pwd -#2. Generate a key pair on the machine where doris manager is deployed -ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa - -#3. Copy the public key to the machine agent01 -scp ~/.ssh/id_rsa.pub root@agent01:~ - -#4. Log in to agent01 and append the public key to authorized_keys -cat ~/id_rsa.pub >> .ssh/authorized_keys - -#5. After doing this, we can log in to agent01 without password on the doris manger machine -ssh agent01@xx.xxx.xx.xx -```` - -In addition, it should be noted that the permissions of the .ssh directory are 700, and the permissions of the authorized_keys and private keys under it are 600. Otherwise, you will not be able to log in without a password due to permission issues. We can see that the known_hosts file will be generated after logging in. At the same time, when starting doris, you need to use a password-free login account. - -When installing a cluster in Doris Manager, just use the private key of the doris manager machine, ie ~/.ssh/id_rsa - -For details, please refer to: https://blog.csdn.net/universe_hao/article/details/52296811 - -#### Specify the installation path - -1. Doris and Doris Manger Agent will be installed in this directory. Make sure this directory is dedicated to Doirs and related components. -2. Specify the Agent startup port, the default is 8001, if there is a conflict, you can customize it. - -### 4 Verify the host - -The system will automatically perform verification according to the host status. When the verification is completed, the Agent will start sending back the heartbeat, and you can click to proceed to the next step. - -![](/images/doris-manager/spacelist-10.png) - -### 5 Verify the cluster - -Verify the cluster quantile instance installation verification, instance dependency verification, and instance startup verification. After the verification is successful, click Next to complete the creation. - -![](/images/doris-manager/spacelist-11.png) - -### 6 Complete access - -Complete the above steps to complete cluster hosting. - -## Self-built http service - -### 1 yum source installation - -1. Installation -yum install -y nginx -2. Start -systemctl start nginx - -### 2 Source installation - -Reference: https://www.runoob.com/linux/nginx-install-setup.html - -### 3 Configuration - -1. Put the doris installation package in the nginx root directory -mv apache-doris-1.1.1-bin-x86.tar.gz /usr/share/nginx/html - -2. Modify ngixn.conf - -```` -location /download { - alias /home/work/nginx/nginx/html/; -} -```` - -Restart ngxin access after modification: -https://host:port/download/apache-doris-1.1.1-bin-x86.tar.gz diff --git a/docs/en/docs/ecosystem/doris-manager/space-management.md b/docs/en/docs/ecosystem/doris-manager/space-management.md deleted file mode 100644 index c6df3098e4..0000000000 --- a/docs/en/docs/ecosystem/doris-manager/space-management.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -{ - "title": "Space management", - "language": "en" -} ---- - - - -# Space management - -If you are Doris Manger's space administrator, you will have permission to set or manage the space, space members, roles, etc. - -## Space - -The space administrator can mainly perform the following operations under the space module: - -- Edit saved space information - -### Edit space information - -If the space information is complete, space administrators can view and edit space-related information here, including space name, space introduction, space administrator, etc. - -![](/images/doris-manager/spacemanagement-1.png) - -## Members - -In the secondary navigation bar of the "Space Management" interface, select "Members" to enter the member management page. This page can view all users in the current space, and you can remove members. - -![](/images/doris-manager/spacemanagement-2.png) - -## Role - -Space administrators can view all roles and role members by clicking the "Roles" button in the navigation bar. New members by default belong to the "Space Member" role and the "Space Admin" role. The default roles are "Space Admin" and "Space Member" and cannot be changed by other administrators. - -![](/images/doris-manager/spacemanagement-3.png) diff --git a/docs/en/docs/ecosystem/doris-manager/system-settings.md b/docs/en/docs/ecosystem/doris-manager/system-settings.md deleted file mode 100644 index bae7c2a16f..0000000000 --- a/docs/en/docs/ecosystem/doris-manager/system-settings.md +++ /dev/null @@ -1,91 +0,0 @@ ---- -{ - "title": "System settings", - "language": "en" -} ---- - - - -# System settings - -The super administrator can mainly perform the following operations under the platform module: - -- Perform relevant operations on platform users -- Have the highest level of authority on the platform - -User permission description - -## users - -### User management under local authentication - -Click the Add User button to create a new user with username and email information. - - Doris Manger will assign a temporary password to the new user. The new user needs to log in with the set username/email and temporary password. After logging in, you can create a new password in "Account Settings". - -![](/images/doris-manager/systemsettings-1.png) - -![](/images/doris-manager/systemsettings-2.png) - - -### Edit User - -Super administrators can manage users, including editing user information, resetting user passwords, and deactivating users. - -#### Edit user information - -Click to select and select "Edit" to modify the user name and email address. If the user mailbox is updated, the user needs to log in with the updated mailbox, and the password will not be updated. - -![](/images/doris-manager/systemsettings-3.png) - -#### reset user password - -Click to select "Reset Password", and after confirming this operation, Doris Manger will reassign a temporary password for the user. The user needs to log in with the set email address and the new temporary password. After logging in, you can go to "Account Settings" Create a new password. - - -#### Deactivate/Activate User - -Click Opt-out user, and after confirming to deactivate the user, the user's status will be changed from active to inactive. Deactivated users will not be able to log in to Doris Manger. - -Click Activate User on the right side of the user to reactivate the user. The user's status will be changed back to enabled and will be able to log in to Doris Manger again. - -Note that super administrators cannot deactivate their own user accounts, and there must be at least one non-deactivated super administrator user in the system. - -![](/images/doris-manager/systemsettings-4.png) - - -## User permission description - -### Super administrator privileges - -| | Create | Edit | Delete | View | -| :---- | :----- | :--- | :----- | :--- | -| User | ✓ | ✓ | ✓ | ✓ | -| Roles | ✓ | ✓ | ✓ | ✓ | -| Space | ✓ | ✓ | ✓ | ✓ | - -### Space administrator permissions - -| | Create | Edit | Delete | View | -| :---- | :----- | :--- | :----- | :--- | -| User | X | X | X | X | -| Roles | X | X | X | ✓ | -| Space | X | ✓ | X | ✓ | diff --git a/docs/images/doris-manager/clustermanagenent-1.png b/docs/images/doris-manager/clustermanagenent-1.png deleted file mode 100644 index 3607a1a93f..0000000000 Binary files a/docs/images/doris-manager/clustermanagenent-1.png and /dev/null differ diff --git a/docs/images/doris-manager/clustermanagenent-2.png b/docs/images/doris-manager/clustermanagenent-2.png deleted file mode 100644 index 90fd0352e2..0000000000 Binary files a/docs/images/doris-manager/clustermanagenent-2.png and /dev/null differ diff --git a/docs/images/doris-manager/clustermanagenent-3.png b/docs/images/doris-manager/clustermanagenent-3.png deleted file mode 100644 index 25c57be833..0000000000 Binary files a/docs/images/doris-manager/clustermanagenent-3.png and /dev/null differ diff --git a/docs/images/doris-manager/clustermanagenent-4.png b/docs/images/doris-manager/clustermanagenent-4.png deleted file mode 100644 index d53b4d3bc5..0000000000 Binary files a/docs/images/doris-manager/clustermanagenent-4.png and /dev/null differ diff --git a/docs/images/doris-manager/clustermanagenent-5.png b/docs/images/doris-manager/clustermanagenent-5.png deleted file mode 100644 index d4c8db4987..0000000000 Binary files a/docs/images/doris-manager/clustermanagenent-5.png and /dev/null differ diff --git a/docs/images/doris-manager/initializing-1.png b/docs/images/doris-manager/initializing-1.png deleted file mode 100644 index a54a416a3f..0000000000 Binary files a/docs/images/doris-manager/initializing-1.png and /dev/null differ diff --git a/docs/images/doris-manager/initializing-2.png b/docs/images/doris-manager/initializing-2.png deleted file mode 100644 index 8ddc4881e2..0000000000 Binary files a/docs/images/doris-manager/initializing-2.png and /dev/null differ diff --git a/docs/images/doris-manager/spacelist-1.png b/docs/images/doris-manager/spacelist-1.png deleted file mode 100644 index 9d978cc43d..0000000000 Binary files a/docs/images/doris-manager/spacelist-1.png and /dev/null differ diff --git a/docs/images/doris-manager/spacelist-10.png b/docs/images/doris-manager/spacelist-10.png deleted file mode 100644 index eda6ab57ad..0000000000 Binary files a/docs/images/doris-manager/spacelist-10.png and /dev/null differ diff --git a/docs/images/doris-manager/spacelist-11.png b/docs/images/doris-manager/spacelist-11.png deleted file mode 100644 index 3162a5b4e5..0000000000 Binary files a/docs/images/doris-manager/spacelist-11.png and /dev/null differ diff --git a/docs/images/doris-manager/spacelist-2.png b/docs/images/doris-manager/spacelist-2.png deleted file mode 100644 index 6772461cce..0000000000 Binary files a/docs/images/doris-manager/spacelist-2.png and /dev/null differ diff --git a/docs/images/doris-manager/spacelist-3.png b/docs/images/doris-manager/spacelist-3.png deleted file mode 100644 index b4c9a5e29a..0000000000 Binary files a/docs/images/doris-manager/spacelist-3.png and /dev/null differ diff --git a/docs/images/doris-manager/spacelist-4.png b/docs/images/doris-manager/spacelist-4.png deleted file mode 100644 index f499ed17db..0000000000 Binary files a/docs/images/doris-manager/spacelist-4.png and /dev/null differ diff --git a/docs/images/doris-manager/spacelist-5.png b/docs/images/doris-manager/spacelist-5.png deleted file mode 100644 index a63e664849..0000000000 Binary files a/docs/images/doris-manager/spacelist-5.png and /dev/null differ diff --git a/docs/images/doris-manager/spacelist-6.png b/docs/images/doris-manager/spacelist-6.png deleted file mode 100644 index f79090a512..0000000000 Binary files a/docs/images/doris-manager/spacelist-6.png and /dev/null differ diff --git a/docs/images/doris-manager/spacelist-7.png b/docs/images/doris-manager/spacelist-7.png deleted file mode 100644 index 0c6d695ea8..0000000000 Binary files a/docs/images/doris-manager/spacelist-7.png and /dev/null differ diff --git a/docs/images/doris-manager/spacelist-8.png b/docs/images/doris-manager/spacelist-8.png deleted file mode 100644 index 7b274bb782..0000000000 Binary files a/docs/images/doris-manager/spacelist-8.png and /dev/null differ diff --git a/docs/images/doris-manager/spacelist-9.png b/docs/images/doris-manager/spacelist-9.png deleted file mode 100644 index b032e6e0fb..0000000000 Binary files a/docs/images/doris-manager/spacelist-9.png and /dev/null differ diff --git a/docs/images/doris-manager/spacemanagement-1.png b/docs/images/doris-manager/spacemanagement-1.png deleted file mode 100644 index 69be20642b..0000000000 Binary files a/docs/images/doris-manager/spacemanagement-1.png and /dev/null differ diff --git a/docs/images/doris-manager/spacemanagement-2.png b/docs/images/doris-manager/spacemanagement-2.png deleted file mode 100644 index 61699bedb4..0000000000 Binary files a/docs/images/doris-manager/spacemanagement-2.png and /dev/null differ diff --git a/docs/images/doris-manager/spacemanagement-3.png b/docs/images/doris-manager/spacemanagement-3.png deleted file mode 100644 index eeb7256d30..0000000000 Binary files a/docs/images/doris-manager/spacemanagement-3.png and /dev/null differ diff --git a/docs/images/doris-manager/systemsettings-1.png b/docs/images/doris-manager/systemsettings-1.png deleted file mode 100644 index f25b8b79bc..0000000000 Binary files a/docs/images/doris-manager/systemsettings-1.png and /dev/null differ diff --git a/docs/images/doris-manager/systemsettings-2.png b/docs/images/doris-manager/systemsettings-2.png deleted file mode 100644 index 60c60480af..0000000000 Binary files a/docs/images/doris-manager/systemsettings-2.png and /dev/null differ diff --git a/docs/images/doris-manager/systemsettings-3.png b/docs/images/doris-manager/systemsettings-3.png deleted file mode 100644 index 0d28b2a864..0000000000 Binary files a/docs/images/doris-manager/systemsettings-3.png and /dev/null differ diff --git a/docs/images/doris-manager/systemsettings-4.png b/docs/images/doris-manager/systemsettings-4.png deleted file mode 100644 index 9647c11f7c..0000000000 Binary files a/docs/images/doris-manager/systemsettings-4.png and /dev/null differ diff --git a/docs/sidebars.json b/docs/sidebars.json index aa01494a29..446fc0a8b1 100644 --- a/docs/sidebars.json +++ b/docs/sidebars.json @@ -223,18 +223,6 @@ "ecosystem/audit-plugin", "ecosystem/cloudcanal", "ecosystem/hive-bitmap-udf", - { - "type": "category", - "label": "Doris Manager", - "items": [ - "ecosystem/doris-manager/compiling-deploying", - "ecosystem/doris-manager/initializing", - "ecosystem/doris-manager/cluster-managenent", - "ecosystem/doris-manager/space-list", - "ecosystem/doris-manager/space-management", - "ecosystem/doris-manager/system-settings" - ] - }, { "type": "category", "label": "SeaTunnel", diff --git a/docs/zh-CN/docs/data-operate/update-delete/sequence-column-manual.md b/docs/zh-CN/docs/data-operate/update-delete/sequence-column-manual.md index 1a255513a3..f488bf011d 100644 --- a/docs/zh-CN/docs/data-operate/update-delete/sequence-column-manual.md +++ b/docs/zh-CN/docs/data-operate/update-delete/sequence-column-manual.md @@ -26,10 +26,12 @@ under the License. # sequence 列 -sequence列目前只支持Uniq模型，Uniq模型主要针对需要唯一主键的场景，可以保证主键唯一性约束，但是由于使用REPLACE聚合方式，在同一批次中导入的数据，替换顺序不做保证，详细介绍可以参考[数据模型](../../data-table/data-model.md)。替换顺序无法保证则无法确定最终导入到表中的具体数据，存在了不确定性。 +Uniq模型主要针对需要唯一主键的场景，可以保证主键唯一性约束，但是由于使用REPLACE聚合方式，在同一批次中导入的数据，替换顺序不做保证，详细介绍可以参考[数据模型](../../data-table/data-model.md)。替换顺序无法保证则无法确定最终导入到表中的具体数据，存在了不确定性。 为了解决这个问题，Doris支持了sequence列，通过用户在导入时指定sequence列，相同key列下，REPLACE聚合类型的列将按照sequence列的值进行替换，较大值可以替换较小值，反之则无法替换。该方法将顺序的确定交给了用户，由用户控制替换顺序。 +sequence列目前只支持Uniq模型。 + ## 适用场景 Sequence列只能在Uniq数据模型下使用。 diff --git a/docs/zh-CN/docs/ecosystem/doris-manager/cluster-managenent.md b/docs/zh-CN/docs/ecosystem/doris-manager/cluster-managenent.md deleted file mode 100644 index bface60439..0000000000 --- a/docs/zh-CN/docs/ecosystem/doris-manager/cluster-managenent.md +++ /dev/null @@ -1,69 +0,0 @@ ---- -{ - "title": "集群管理", - "language": "zh-CN" -} ---- - - - -# 集群管理 - -超级管理员和空间管理员在集群模块下主要可进行如下操作： - -- 查看集群概览 -- 查看节点列表 -- 编辑参数配置 - -## 集群概览 - -### 查看集群基本信息 - -集群功能，展示以集群为粒度的监控面板。 - -进入首页，点击导航栏中的“集群”，进入集群功能。 - -![](/images/doris-manager/clustermanagenent-1.png) - -运维监控面板提供集群的各类性能监控指标，供用户洞察集群状态。用户可以在右上角通过按钮控制集群的启动和停止操作。 - -### 查看集群资源使用量 - -用户可以通过饼图查看磁盘使用率，以及查看数据库的数量等。 - -## 节点列表 - -展示集群中FE节点、BE节点和Broker相关信息。 -提供包括节点ID、节点类型、主机IP以及节点状态字段。 - -![](/images/doris-manager/clustermanagenent-2.png) - -## 参数配置 - -参数配置提供参数名称、参数类型、参数值类型、热生效和操作字段。 - -![](/images/doris-manager/clustermanagenent-3.png) - -- **操作**：点击“编辑”按钮，可编辑修改对应配置值，可以选择对应的生效方式；点击“查看当前值”按钮，可查看主机IP对应当前值 - -![](/images/doris-manager/clustermanagenent-4.png) - -![](/images/doris-manager/clustermanagenent-5.png) - diff --git a/docs/zh-CN/docs/ecosystem/doris-manager/compiling-deploying.md b/docs/zh-CN/docs/ecosystem/doris-manager/compiling-deploying.md deleted file mode 100644 index 9ca82cc8df..0000000000 --- a/docs/zh-CN/docs/ecosystem/doris-manager/compiling-deploying.md +++ /dev/null @@ -1,124 +0,0 @@ ---- -{ - "title": "编译与部署", - "language": "zh-CN" -} ---- - - - -# 编译与部署 - -## 编译 -直接在manager路径下的build.sh脚本 -```shell -cd incubator-doris-manager -sh build.sh -``` -编译完成后会在manager路径下生成安装包output目录，目录结构如下 -```text -├── agent //agent 目录 -│ ├── bin -│ │ ├── agent_start.sh -│ │ ├── agent_stop.sh -│ │ └── download_doris.sh -│ ├── config -│ │ └── application.properties -│ └── lib -│ └── dm-agent.jar -└── server //server 目录 - ├── bin - │ ├── start_manager.sh //Doris Manager启动脚本 - │ └── stop_manager.sh //Doris Manager停止脚本 - ├── conf - │ └── manager.conf //Doris Manager配置文件 - ├── lib - │ └── doris-manager.jar //Doris Manager的运行包doris-manager.jar - └── web-resource -``` - -## 运行 -### 1 配置 -修改配置文件`server/conf/manager.conf`，重点关注的配置项内容如下： -```$xslt -服务的启动http端口 -STUDIO_PORT=8080 - -后端数据存放的数据库的类型，包括mysql/h2/postgresql.默认是支持mysql -MB_DB_TYPE=mysql - -数据库连接信息 -如果是配置的h2类型数据库，就不需要配置这些信息，会把数据以本地文件存放在本地 -h2数据文件存放路径，默认直接存放在当前路径 -H2_FILE_PATH= - -如果是mysql/postgresql就需要配置如下连接信息 -数据库地址 -MB_DB_HOST= - -数据库端口 -MB_DB_PORT=3306 - -数据库访问端口 -MB_DB_USER= - -数据库访问密码 -MB_DB_PASS= - -数据库的database名称 -MB_DB_DBNAME= - -服务运行的路径，默认直接存放在当前运行路径的log文件夹中 -LOG_PATH= - -web容器的等待队列长度，默认100。队列也做缓冲池用，但也不能无限长，不但消耗内存，而且出队入队也消耗CPU -WEB_ACCEPT_COUNT=100 - -Web容器的最大工作线程数，默认200。（一般是CPU核数*200） -WEB_MAX_THREADS=200 - -Web容器的最小工作空闲线程数，默认10。（适当增大一些，以便应对突然增长的访问量） -WEB_MIN_SPARE_THREADS=10 - -Web容器的最大连接数，默认10000。（适当增大一些，以便应对突然增长的访问量） -WEB_MAX_CONNECTIONS=10000 - -访问数据库连接池最大连接数量，默认为10 -DB_MAX_POOL_SIZE=20 - -访问数据库连接池最小空闲连接数，默认为10 -DB_MIN_IDLE=10 -``` - -### 2 启动 -配置修改完成后，启动doris manger -```$xslt -cd server -sh bin/start_manager.sh -``` -查看logs中的日志即可判断程序是否启动成功 - -### 3 使用 -Doris Manager预设了一个超级管理员用户，信息如下： -```$xslt -用户名: Admin -密码: Admin@123 -``` -为确保使用安全，登陆后请修改密码！ diff --git a/docs/zh-CN/docs/ecosystem/doris-manager/initializing.md b/docs/zh-CN/docs/ecosystem/doris-manager/initializing.md deleted file mode 100644 index aaeb9966ff..0000000000 --- a/docs/zh-CN/docs/ecosystem/doris-manager/initializing.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -{ - "title": "初始化", - "language": "zh-CN" -} ---- - - - -# 初始化 - -完成部署之后，超级管理员需要完成本地初始化。 - -## 管理用户 - -初始化第一步为管理用户，主要完成对认证方式的选择和配置。目前 Doris Manger 支持本地用户认证。 - -![](/images/doris-manager/initializing-1.png) - -### 本地用户认证 - -本地用户认证是 Doris Manger 自带的用户系统。通过填入用户名、邮箱和密码可完成用户注册，用户的新增、信息修改、删除和权限关系均在本地内完成。 - -![](/images/doris-manager/initializing-2.png) - -此时已经完成了本地初始化过程。超级管理员可以创建空间，空间管理员进入空间，进行空间管理，添加邀请用户进入空间进行数据分析等工作。 \ No newline at end of file diff --git a/docs/zh-CN/docs/ecosystem/doris-manager/space-list.md b/docs/zh-CN/docs/ecosystem/doris-manager/space-list.md deleted file mode 100644 index 8f1ec2ea54..0000000000 --- a/docs/zh-CN/docs/ecosystem/doris-manager/space-list.md +++ /dev/null @@ -1,231 +0,0 @@ ---- -{ - "title": "空间列表", - "language": "zh-CN" -} ---- - - - -# 空间列表 - -超级管理员在空间列表主要可进行如下操作： - -- 进行新建集群和集群托管操作 - -- 未完成空间的恢复和删除操作 - -- 已完成空间的删除操作 - -空间管理员在空间列表主要可进行如下操作： - -- 查看有权限的空间信息 - -## 已完成空间 - -超级管理员可以通过空间名称右侧按钮对已完成空间进行操作。空间管理员可以点击进入空间，对空间内的集群或数据进行管理操作。 - -![](/images/doris-manager/spacelist-1.png) - -## 未完成空间 - -Doris Manger 提供了空间创建流程的草稿保存功能，用以记录未完成的空间创建流程。超级管理员可从通过切换tab页查看未完成空间列表，进行恢复或是删除操作。 - -![](/images/doris-manager/spacelist-2.png) - -# 新建空间 - -新建空间包括新建集群和集群托管两种方式。 - -## 新建集群 - -### 1 注册空间 - -空间信息包括空间名称、空间简介、选择空间管理员。 - -空间名称、管理员为必填/选字段。 - -![](/images/doris-manager/spacelist-3.png) - -### 2 添加主机 - -![](/images/doris-manager/spacelist-4.png) - -#### 配置SSH免登陆 - -Doris Manager 在安装时需要分发Agent安装包，故需要在待安装Doris的服务器(agent01)配置SSH免登陆。 - -```shell -#1.登录服务器，需要使用manager和agent账号保持一致 -su - xxx -pwd -#2.在部署doris manager机器上生成密钥对 -ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa - -#3.将公钥拷贝到机器agent01上 -scp ~/.ssh/id_rsa.pub root@agent01:~ - -#4.登录agent01，将公钥追加到authorized_keys -cat ~/id_rsa.pub >> .ssh/authorized_keys - -#5.这样做完之后我们就可以在doris manger机器免密码登录agent01 -ssh agent01@xx.xxx.xx.xx -``` -详细可参考：https://blog.csdn.net/universe_hao/article/details/52296811 - -另外需要注意，.ssh目录的权限为700，其下文件authorized_keys和私钥的权限为600。否则会因为权限问题导致无法免密码登录。我们可以看到登陆后会有known_hosts文件生成。同时启动doris时需要使用免密码登录的账号。 - - -在Doris Manager 安装集群第二步添加节点时，使用部署doris manager机器的私钥即可，即~/.ssh/id_rsa(注意：包括密钥文件的头尾) - -#### 主机列表 -输入主机IP添加新的主机，也可通过批量添加。 - -### 3 安装选项 - -#### 获取安装包 - -1. 代码包路径 - - 通过Doris Manager 进行集群部署时，需要提供已编译好的 Doris 安装包，您可以通过 Doris 源码自行编译. - -`Doris Manager 将通过 http 方式拉取Doris安装包，若您需要自建 http 服务，请参考文档底部-自建http服务`。 - -#### 指定安装路径 - -1. Doris与Doris Manger Agent将安装至该目录下。请确保该目录为Doirs以及相关组件专用。 -2. 指定Agent启动端口，默认为8001，若有冲突，可自定义。 - -### 4 校验主机 - -系统会根据主机状态自动进行校验，当校验完成时既Agent启动回传心跳，可点击进行下一步。 - -![](/images/doris-manager/spacelist-5.png) - -### 5 规划节点 - -点击分配节点按钮，对主机进行FE/BE/Broker节点的规划。 - -![](/images/doris-manager/spacelist-6.png) - -### 6 配置参数 - -对上一步规划的节点进行配置参数，可以使用默认值也可以打开自定义配置开关对配置进行自定义。 - -### 7 部署集群 - -系统会根据主机安装进度状态自动进行校验，当校验完成时既启动节点并回传心跳，可点击进行下一步。 - -![](/images/doris-manager/spacelist-7.png) - -### 8 完成创建 - -完成以上步骤即可完成新建集群。 - -![](/images/doris-manager/spacelist-8.png) - -## 集群托管 - -### 1 注册空间 - -空间信息包括空间名称、空间简介、选择空间管理员。 - -空间名称、管理员为必填/选字段。 - -### 2 连接集群 - -集群信息包括集群地址、HTTP端口、JDBC端口、集群用户名和集群密码。用户可根据自身集群信息进行填写。 - -点击链接测试按钮进行测试。 - -### 3 托管选项 - -![](/images/doris-manager/spacelist-9.png) - -#### 配置SSH免登陆 - -Doris Manager 在安装时需要分发Agent安装包，故需要在待安装Doris的服务器(agent01)配置SSH免登陆。 - -```shell -#1.登录服务器，需要使用manger和agent账号保持一致 -su - xxx -pwd -#2.在部署doris manager机器上生成密钥对 -ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa - -#3.将公钥拷贝到机器agent01上 -scp ~/.ssh/id_rsa.pub root@agent01:~ - -#4.登录agent01，将公钥追加到authorized_keys -cat ~/id_rsa.pub >> .ssh/authorized_keys - -#5.这样做完之后我们就可以在doris manger机器免密码登录agent01 -ssh agent01@xx.xxx.xx.xx -``` - -另外需要注意，.ssh目录的权限为700，其下文件authorized_keys和私钥的权限为600。否则会因为权限问题导致无法免密码登录。我们可以看到登陆后会有known_hosts文件生成。同时启动doris时需要使用免密码登录的账号。 - -在Doris Manager 安装集群时，使用部署doris manager机器的私钥即可，即~/.ssh/id_rsa - -详细可参考：https://blog.csdn.net/universe_hao/article/details/52296811 - -#### 指定安装路径 - -1. Doris与Doris Manger Agent将安装至该目录下。请确保该目录为Doirs以及相关组件专用。 -2. 指定Agent启动端口，默认为8001，若有冲突，可自定义。 - -### 4 校验主机 - -系统会根据主机状态自动进行校验，当校验完成时既Agent启动回传心跳，可点击进行下一步。 - -![](/images/doris-manager/spacelist-10.png) - -### 5 校验集群 -校验集群分位实例安装校验、实例依赖校验、实例启动校验，校验成功后点击下一步即可完成创建。 - -![](/images/doris-manager/spacelist-11.png) - -### 6 完成接入 -完成以上步骤即可完成集群托管。 - -## 自建http服务 - -### 1 yum源安装 -1.安装 -yum install -y nginx -2.启动 -systemctl start nginx - -### 2 源码安装 -可参考：https://www.runoob.com/linux/nginx-install-setup.html - -### 3 配置 - -1.将doris安装包放置nginx根目录 -mv apache-doris-1.1.1-bin-x86.tar.gz /usr/share/nginx/html - -2.修改ngixn.conf -``` -location /download { - alias /home/work/nginx/nginx/html/; - } -``` -修改后重启ngxin访问 ： -https://host:port/download/apache-doris-1.1.1-bin-x86.tar.gz diff --git a/docs/zh-CN/docs/ecosystem/doris-manager/space-management.md b/docs/zh-CN/docs/ecosystem/doris-manager/space-management.md deleted file mode 100644 index 2e7ff668f5..0000000000 --- a/docs/zh-CN/docs/ecosystem/doris-manager/space-management.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -{ - "title": "空间管理", - "language": "zh-CN" -} ---- - - - -# 空间管理 - -若您为 Doris Manger 的空间管理员，您将有权限对于空间、空间成员、角色等进行设置或管理。 - -## 空间 - -空间管理员在空间模块下主要可进行如下操作： - -- 编辑已经保存的空间信息 - -### 编辑空间信息 - -如果空间信息已经完备，空间管理员可以在此查看并编辑空间相关信息，包括空间名称、空间简介、空间管理员等等。 - -![](/images/doris-manager/spacemanagement-1.png) - -## 成员 - -在“空间管理”界面的二级导航栏中选择“成员”，进入成员管理页面。此页面可以查看当前空间下的所有用户，您可以对成员进行移除操作。 - -![](/images/doris-manager/spacemanagement-2.png) - -## 角色 - -空间管理员可以通过单击导航栏“角色”按钮，查看所有角色和角色成员。新成员默认属于"空间成员"角色和"空间管理员"角色。默认角色有"空间管理员"和"空间成员"且不可被其他管理员更改。 - -![](/images/doris-manager/spacemanagement-3.png) diff --git a/docs/zh-CN/docs/ecosystem/doris-manager/system-settings.md b/docs/zh-CN/docs/ecosystem/doris-manager/system-settings.md deleted file mode 100644 index 3d79650561..0000000000 --- a/docs/zh-CN/docs/ecosystem/doris-manager/system-settings.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -{ - "title": "系统设置", - "language": "zh-CN" -} ---- - - - -# 系统设置 - -超级管理员在平台模块下主要可进行如下操作： - -- 对平台用户进行相关操作 -- 拥有平台最高级权限 - -用户权限说明 - -## 用户 - -### 本地认证下用户管理 - -单击添加用户按钮，可以通过用户名与邮箱信息创建新用户。 - - Doris Manger 将会为新用户分配临时密码，新用户需要使用所设定的用户名/邮箱和临时密码登录，登录后可在“账户设置”中创建新密码。 - -![](/images/doris-manager/systemsettings-1.png) - -![](/images/doris-manager/systemsettings-2.png) - - -### 编辑用户 - -超级管理员可以对用户进行管理，包括编辑用户信息、重置用户密码、停用用户等操作。 - -#### 编辑用户信息 - -点击选择选择“编辑”可以修改用户名、邮箱。若更新用户邮箱，则用户需要使用更新后的邮箱进行登录，密码不会被更新。 - -![](/images/doris-manager/systemsettings-3.png) - -#### 重置用户密码 - -点击选择“重置密码”，确认执行此操作后， Doris Manger 将会为该用户重新分配临时密码，用户需要使用所设定的邮箱和新的临时密码登录，登录后可在“账户设置”中创建新密码。 - - -#### 停用/激活用户 - -点击选择停用用户，确认停用该用户后，该用户状态将由启用变更为停用。已停用用户将无法登录 Doris Manger 。 - -点击用户右侧的激活用户，可以重新激活该用户。该用户状态将重新变更为启用，能够再次登录 Doris Manger 。 - -注意，超级管理员不能停用自己的用户账户，系统内至少要有一位非停用状态的超级管理员用户。 - -![](/images/doris-manager/systemsettings-4.png) - - -## 用户权限说明 - -### 超级管理员权限 -| | 创建 | 编辑 | 删除 | 查看 | -|:----------|:----------|:----------|:----------|:----------| -| 用户 | ✓ | ✓ | ✓ | ✓ | -| 角色 | ✓ | ✓ | ✓ | ✓ | -| 空间 | ✓ | ✓ | ✓ | ✓ | - -### 空间管理员权限 -| | 创建 | 编辑 | 删除 | 查看 | -|:----------|:----------|:----------|:----------|:----------| -| 用户 | X | X | X | X | -| 角色 | X | X | X | ✓ | -| 空间 | X | ✓ | X | ✓ | -