* [doris-1008] support backup and restore directly to cloud storage via aws s3 protocol * Internal][S3DirectAccess] Support backup,restore,load,export directlyconnect to s3 1. Support load and export data from/to s3 directly. 2. Add a config to auto convert broker access to s3 acces when available Change-Id: Iac96d4b3670776708bc96a119ff491db8cb4cde7 (cherry picked from commit 2f03832ca52221cc7436069b96c45c48c4bc7201) * [Internal][S3DirectAccess] File path glob compatible with broker Change-Id: Ie55e07a547aa22c6fa8d432ca926216c10384e68 (cherry picked from commit d4fb25544c0dc06d23e1ada571ec3f8edd4ba56f) * [internal] [doris-1008] fix log4j class not found Change-Id: I468176aca0d821383c74ee658d461aba9e7d5be3 (cherry picked from commit 029adaa9d6ded8503acbd6644c1519456f3db232) * add poms Co-authored-by: yangzhengguo01 <yangzhengguo01@baidu.com>
3.2 KiB
3.2 KiB
title, language
| title | language |
|---|---|
| CREATE REPOSITORY | en |
CREATE REPOSITORY
Description
This statement is used to create the warehouse. The warehouse is used for backup or recovery. Only root or superuser users can create warehouses.
Grammar:
CREATE [READ ONLY] REPOSITORY `repo_name`
WITH [BROKER `broker_name`|S3]
ON LOCATION `repo_location`
PROPERTIES ("key"="value", ...);
Explain:
1. The creation of warehouses depends on existing brokers, or use aws s3 protocl to connet cloud storage directly.
2. If it is a read-only warehouse, it can only be restored on the warehouse. If not, you can backup and restore operations.
3. According to the different types of broker or S3, PROPERTIES is different, see the example.
example
1. Create a warehouse named bos_repo, which relies on BOS broker "bos_broker", and the data root directory is: bos://palo_backup.
CREATE REPOSITORY `bos_repo`
WITH BROKER `bos_broker`
ON LOCATION "bos://palo_backup"
PROPERTIES
(
"bos_endpoint" ="http://gz.bcebos.com",
"bos_accesskey" = "bos_accesskey",
"bos_secret_accesskey"="bos_accesskey"
);
2. Create the same warehouse as in Example 1, but with read-only attributes:
CREATE READ ONLY REPOSITORY `bos_repo`
WITH BROKER `bos_broker`
ON LOCATION "bos://palo_backup"
PROPERTIES
(
"bos_endpoint" ="http://gz.bcebos.com",
"bos_accesskey" = "bos_accesskey",
"bos_secret_accesskey"="bos_secret_accesskey"
);
3. Create a warehouse named hdfs_repo, which relies on Baidu HDFS broker "hdfs_broker", and the data root directory is: hdfs://hadoop-name-node:54310/path/to/repo./
CREATE REPOSITORY `hdfs_repo`
WITH BROKER `hdfs_broker`
ON LOCATION "hdfs://hadoop-name-node:54310/path/to/repo/"
PROPERTIES
(
"Username" = "User"
"password" = "password"
);
4. 创建名为 s3_repo 的仓库,直接链接云存储,而不通过broker.
CREATE REPOSITORY `s3_repo`
WITH S3
ON LOCATION "s3://s3-repo"
PROPERTIES
(
"AWS_ENDPOINT" = "http://s3-REGION.amazonaws.com",
"AWS_ACCESS_KEY" = "AWS_ACCESS_KEY",
"AWS_SECRET_KEY"="AWS_SECRET_KEY",
"AWS_REGION" = "REGION"
);
keyword
CREATE REPOSITORY