Files

qiye 3b8d48f08b [feature-wip](iceberg) Step1: Support create Iceberg external table (#7391 )

Close related #7389

Support create Iceberg external table in Doris. 

This is the first step to support Iceberg external table.

### Create Iceberg external table
This pr describes two ways to create Iceberg external tables. Both ways do not require explicitly specifying column definitions, Doris automatically converts them based on Iceberg's column definitions.

1. Create an Iceberg external table directly

```sql
    CREATE [EXTERNAL] TABLE table_name 
    ENGINE = ICEBERG
    [COMMENT "comment"]
    PROPERTIES (
    "iceberg.database" = "iceberg_db_name",
    "iceberg.table" = "icberg_table_name",
    "iceberg.hive.metastore.uris"  =  "thrift://192.168.0.1:9083",
    "iceberg.catalog.type"  =  "HIVE_CATALOG"
    );
```

2. Create an Iceberg database and automatically create all the tables under that db.

```sql
    CREATE DATABASE db_name 
    [COMMENT "comment"]
    PROPERTIES (
    "iceberg.database" = "iceberg_db_name",
    "iceberg.hive.metastore.uris" = "thrift://192.168.0.1:9083",
    "iceberg.catalog.type" = "HIVE_CATALOG"
    );
```

### Show table creation

1. For individual tables you can view them with `help show create table`.

```sql 
mysql> show create table iceberg_db.logs_1;
+--------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table  | Create Table                                                                                                                                                                                                                                                                                                                                                 |
+--------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| logs_1 | CREATE TABLE `logs_1` (
  `level` varchar(-1) NOT NULL COMMENT "null",
  `event_time` datetime NOT NULL COMMENT "null",
  `message` varchar(-1) NOT NULL COMMENT "null"
) ENGINE=ICEBERG
COMMENT "ICEBERG"
PROPERTIES (
"iceberg.database" = "doris",
"iceberg.table" = "logs_1",
"iceberg.hive.metastore.uris"  =  "thrift://10.10.10.10:9087",
"iceberg.catalog.type"  =  "HIVE_CATALOG"
) |
+--------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
```

2. For Iceberg database, you can view it with `help show table creation`.

```sql
mysql> show table creation from iceberg_db;
+--------+---------+---------------------+---------------------------------------------------------+
| Table  | Status  | Create Time         | Error Msg                                               |
+--------+---------+---------------------+---------------------------------------------------------+
| logs   | fail    | 2021-12-14 13:50:10 | Cannot convert unknown type to Doris type: list<string> |
| logs_1 | success | 2021-12-14 13:50:10 |                                                         |
+--------+---------+---------------------+---------------------------------------------------------+
2 rows in set (0.00 sec)
```

  This is a new syntax.
  
  Show table creation records in Iceberg database:
  
  Syntax:
  ```sql
      SHOW TABLE CREATION [FROM db] [LIKE mask]
  ```

2022-01-27 10:22:47 +08:00

4.3 KiB

Raw Blame History

title, language

title	language
SHOW TABLE CREATION	en

SHOW TABLE CREATION

Description

This statement is used to show the execution of the specified Iceberg Database table creation task
Syntax.
    SHOW TABLE CREATION [FROM db_name] [LIKE table_name_wild];
    
Description.
    1. Usage Notes
        1) If db_name is not specified, the current default db is used.
        2) If you use LIKE, it will match the table creation task with table_name_wild in the table name
    2. The meaning of each column
        1) Database: the name of the database
        2) Table: the name of the table to be created
        3) Status: the creation status of the table, `success`/`fail`.
        4) CreateTime: the time to perform the task of creating the table
        5) Error Msg: Error message of the failed table creation, or empty if it succeeds.

example

1. Show all the table creation tasks in the default Iceberg db
    SHOW TABLE CREATION;

    mysql> show table creation;
    +----------------------------+--------+---------+---------------------+----------------------------------------------------------+
    | Database                   | Table  | Status  | Create Time         | Error Msg                                                |
    +----------------------------+--------+---------+---------------------+----------------------------------------------------------+
    | default_cluster:iceberg_db | logs_1 | success | 2022-01-24 19:42:45 |                                                          |
    | default_cluster:iceberg_db | logs   | fail    | 2022-01-24 19:42:45 | Cannot convert Iceberg type[list<string>] to Doris type. |
    +----------------------------+--------+---------+---------------------+----------------------------------------------------------+

2. Show the table creation tasks in the specified Iceberg db
    SHOW TABLE CREATION FROM example_db;

    mysql> show table creation from iceberg_db;
    +----------------------------+--------+---------+---------------------+----------------------------------------------------------+
    | Database                   | Table  | Status  | Create Time         | Error Msg                                                |
    +----------------------------+--------+---------+---------------------+----------------------------------------------------------+
    | default_cluster:iceberg_db | logs_1 | success | 2022-01-24 19:42:45 |                                                          |
    | default_cluster:iceberg_db | logs   | fail    | 2022-01-24 19:42:45 | Cannot convert Iceberg type[list<string>] to Doris type. |
    +----------------------------+--------+---------+---------------------+----------------------------------------------------------+
    
3. Show table creation tasks for the specified Iceberg db with the string "log" in the table name
    SHOW TABLE CREATION FROM example_db LIKE '%log%';
    
    mysql> show table creation from iceberg_db like "%1";
    +----------------------------+--------+---------+---------------------+-----------+
    | Database                   | Table  | Status  | Create Time         | Error Msg |
    +----------------------------+--------+---------+---------------------+-----------+
    | default_cluster:iceberg_db | logs_1 | success | 2022-01-24 19:42:45 |           |
    +----------------------------+--------+---------+---------------------+-----------+

keyword

SHOW,TABLE CREATION

4.3 KiB Raw Blame History

SHOW TABLE CREATION

Description

example

keyword

4.3 KiB

Raw Blame History