database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

Support table comment and column comment for view (#1799)

Browse Source
This commit is contained in:
Mingyu Chen
2019-09-18 09:45:28 +08:00
committed by ZHAO Chun
parent 3f63bde5cb
commit 714dca8699
27 changed files with 374 additions and 140 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
docs/documentation/cn/sql-reference/sql-statements/Administration/SHOW FULL COLUMNS.md Normal file
View File

@ -0,0 +1,14 @@
# SHOW FULL COLUMNS
## description
该语句用于指定表的列信息
语法：
SHOW FULL COLUMNS FROM tbl;
## example
1. 查看指定表的列信息
SHOW FULL COLUMNS FROM tbl;
## keyword
SHOW,TABLE,STATUS

22
docs/documentation/cn/sql-reference/sql-statements/Administration/SHOW TABLE STATUS.md Normal file
View File

@ -0,0 +1,22 @@
# SHOW TABLE STATUS
## description
该语句用于查看 Table 的一些信息。
语法：
SHOW TABLE STATUS
[FROM db] [LIKE "pattern"]
说明：
1. 该语句主要用于兼容 MySQL 语法，目前仅显示 Comment 等少量信息
## example
1. 查看当前数据库下所有表的信息
SHOW TABLE STATUS;
2. 查看指定数据库下，名称包含 example 的表的信息
SHOW TABLE STATUS FROM db LIKE "%example%";
## keyword
SHOW,TABLE,STATUS

6
docs/documentation/cn/sql-reference/sql-statements/Data Definition/CREATE TABLE.md
View File

@ -6,10 +6,11 @@
(column_definition1[, column_definition2, ...])
[ENGINE = [olap|mysql|broker]]
[key_desc]
[COMMENT "table comment"];
[partition_desc]
[distribution_desc]
[PROPERTIES ("key"="value", ...)];
[BROKER PROPERTIES ("key"="value", ...)];
[PROPERTIES ("key"="value", ...)]
[BROKER PROPERTIES ("key"="value", ...)]
1. column_definition
语法：
@ -191,6 +192,7 @@
)
ENGINE=olap
AGGREGATE KEY(k1, k2)
COMMENT "my first doris table"
DISTRIBUTED BY HASH(k1) BUCKETS 32
PROPERTIES ("storage_type"="column");

18
docs/documentation/cn/sql-reference/sql-statements/Data Definition/CREATE VIEW.md
View File

@ -3,7 +3,8 @@
该语句用于创建一个逻辑视图
语法：
CREATE VIEW [IF NOT EXISTS]
[db_name.]view_name (column1[, column2, ...])
[db_name.]view_name
(column1[ COMMENT "col comment"][, column2, ...])
AS query_stmt
说明：
@ -12,10 +13,25 @@
## example
1. 在 example_db 上创建视图 example_view
CREATE VIEW example_db.example_view (k1, k2, k3, v1)
AS
SELECT c1 as k1, k2, k3, SUM(v1) FROM example_table
WHERE k1 = 20160112 GROUP BY k1,k2,k3;
2. 创建一个包含 comment 的 view
CREATE VIEW example_db.example_view
(
k1 COMMENT "first key",
k2 COMMENT "second key",
k3 COMMENT "third key",
v1 COMMENT "first value"
)
COMMENT "my first view"
AS
SELECT c1 as k1, k2, k3, SUM(v1) FROM example_table
WHERE k1 = 20160112 GROUP BY k1,k2,k3;
## keyword
CREATE,VIEW

16
docs/documentation/en/sql-reference/sql-statements/Administration/SHOW FULL COLUMNS_EN.md Normal file
View File

@ -0,0 +1,16 @@
# SHOW FULL COLUMNS
## description
This statement is used to view some information about columns of a table.
Syntax：
SHOW FULL COLUMNS FROM tbl;
## example
1. View the column information of specified table
SHOW FULL COLUMNS FROM tbl;
## keyword
SHOW,FULL,COLUMNS

29
docs/documentation/en/sql-reference/sql-statements/Administration/SHOW TABLE STATUS_EN.md Normal file
View File

@ -0,0 +1,29 @@
# SHOW TABLE STATUS
## description
This statement is used to view some information about Table.
Syntax:
SHOW TABLE STATUS
[FROM db] [LIKE "pattern"]
Explain:
1. This statement is mainly used to be compatible with MySQL grammar. At present, only a small amount of information such as Comment is displayed.
## Example
1. View the information of all tables under the current database
SHOW TABLE STATUS;
2. View the information of the table whose name contains example in the specified database
SHOW TABLE STATUS FROM DB LIKE "% example%";
## Keyword
SHOW,TABLE,STATUS

18
docs/documentation/en/sql-reference/sql-statements/Data Definition/CREATE TABLE_EN.md
View File

@ -3,14 +3,15 @@
### Syntax
CREATE [EXTERNAL] TABLE [IF NOT EXISTS] [database.]table_name
(column_definition1[, column_definition2, ...])
[ENGINE = [olap|mysql|broker]]
[key_desc]
[partition_desc]
[distribution_desc]
[PROPERTIES ("key"="value", ...)];
[BROKER PROPERTIES ("key"="value", ...)];
CREATE [EXTERNAL] TABLE [IF NOT EXISTS] [database.]table_name
(column_definition1[, column_definition2, ...])
[ENGINE = [olap|mysql|broker]]
[key_desc]
[COMMENT "table comment"]
[partition_desc]
[distribution_desc]
[PROPERTIES ("key"="value", ...)]
[BROKER PROPERTIES ("key"="value", ...)];
1. column_definition
@ -222,6 +223,7 @@ CREATE [EXTERNAL] TABLE [IF NOT EXISTS] [database.]table_name
)
ENGINE=olap
AGGREGATE KEY(k1, k2)
COMMENT "my first doris table"
DISTRIBUTED BY HASH(k1) BUCKETS 32
PROPERTIES ("storage_type"="column");
```

48
docs/documentation/en/sql-reference/sql-statements/Data Definition/CREATE VIEW_EN.md
View File

@ -1,22 +1,42 @@
# CREATE VIEW
## Description
This statement is used to create a logical view
Grammar:
CREATE VIEW [IF NOT EXISTS]
[db_name.]view_name (column1[, column2, ...])
AS query
This statement is used to create a logical view
Grammar:
CREATE VIEW [IF NOT EXISTS]
[db_name.]view_name
(column1[ COMMENT "col comment"][, column2, ...])
AS query_stmt
Explain:
1. Views are logical views without physical storage. All queries on views are equivalent to sub-queries corresponding to views.
2. Query_stmt is arbitrarily supported SQL
Explain:
1. Views are logical views without physical storage. All queries on views are equivalent to sub-queries corresponding to views.
2. query_stmt is arbitrarily supported SQL.
## example
1. Create view example_view on example_db
CREATE VIEW example_db.example_view (k1, k2, k3, v1)
AS
SELECT c1 as k1, k2, k3, SUM(v1) FROM example_table
WHERE k1 = 20160112 GROUP BY k1,k2,k3;
1. Create view example_view on example_db
CREATE VIEW example_db.example_view (k1, k2, k3, v1)
AS
SELECT c1 as k1, k2, k3, SUM(v1) FROM example_table
WHERE k1 = 20160112 GROUP BY k1,k2,k3;
2. Create view with comment
CREATE VIEW example_db.example_view
(
k1 COMMENT "first key",
k2 COMMENT "second key",
k3 COMMENT "third key",
v1 COMMENT "first value"
)
COMMENT "my first view"
AS
SELECT c1 as k1, k2, k3, SUM(v1) FROM example_table
WHERE k1 = 20160112 GROUP BY k1,k2,k3;
## keyword
CREATE,VIEW
CREATE,VIEW

40
fe/src/main/cup/sql_parser.cup
View File

@ -383,6 +383,8 @@ nonterminal String opt_system;
nonterminal String opt_cluster;
nonterminal BrokerDesc opt_broker;
nonterminal List<String> opt_col_list, col_list, opt_dup_keys, opt_columns_from_path;
nonterminal List<ColWithComment> opt_col_with_comment_list, col_with_comment_list;
nonterminal ColWithComment col_with_comment;
nonterminal List<String> opt_partitions, partitions;
nonterminal List<Expr> opt_col_mapping_list;
nonterminal ColumnSeparator opt_field_term, column_separator;
@ -877,12 +879,13 @@ create_stmt ::=
| KW_CREATE opt_external:isExternal KW_TABLE opt_if_not_exists:ifNotExists table_name:name
LPAREN column_definition_list:columns RPAREN opt_engine:engineName
opt_keys:keys
opt_comment:tableComment
opt_partition:partition
opt_distribution:distribution
opt_properties:tblProperties
opt_ext_properties:extProperties
{:
RESULT = new CreateTableStmt(ifNotExists, isExternal, name, columns, engineName, keys, partition, distribution, tblProperties, extProperties);
RESULT = new CreateTableStmt(ifNotExists, isExternal, name, columns, engineName, keys, partition, distribution, tblProperties, extProperties, tableComment);
:}
/* User */
| KW_CREATE KW_USER opt_if_not_exists:ifNotExists grant_user:user opt_user_role:userRole
@ -890,9 +893,9 @@ create_stmt ::=
RESULT = new CreateUserStmt(ifNotExists, user, userRole);
:}
| KW_CREATE KW_VIEW opt_if_not_exists:ifNotExists table_name:viewName
opt_col_list:columns KW_AS query_stmt:view_def
opt_col_with_comment_list:columns opt_comment:comment KW_AS query_stmt:view_def
{:
RESULT = new CreateViewStmt(ifNotExists, viewName, columns, view_def);
RESULT = new CreateViewStmt(ifNotExists, viewName, columns, comment, view_def);
:}
/* cluster */
| KW_CREATE KW_CLUSTER ident:name opt_properties:properties KW_IDENTIFIED KW_BY STRING_LITERAL:password
@ -1133,6 +1136,37 @@ opt_col_list ::=
:}
;
opt_col_with_comment_list ::=
{:
RESULT = null;
:}
| LPAREN col_with_comment_list:colList RPAREN
{:
RESULT = colList;
:}
;
col_with_comment_list ::=
col_with_comment:col
{:
ArrayList<ColWithComment> list = new ArrayList<ColWithComment>();
list.add(col);
RESULT = list;
:}
| col_with_comment_list:list COMMA col_with_comment:col
{:
list.add(col);
RESULT = list;
:}
;
col_with_comment ::=
ident:col opt_comment:comment
{:
RESULT = new ColWithComment(col, comment);
:}
;
col_list ::=
KW_COLUMNS LPAREN ident_list:colList RPAREN
{:

55
fe/src/main/java/org/apache/doris/analysis/ColWithComment.java Normal file
View File

@ -0,0 +1,55 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.analysis;
import org.apache.doris.common.AnalysisException;
import org.apache.doris.common.FeNameFormat;
import com.google.common.base.Strings;
public class ColWithComment {
private String colName;
private String comment;
public ColWithComment(String colName, String comment) {
this.colName = colName;
this.comment = Strings.nullToEmpty(comment);
}
public void analyze() throws AnalysisException {
FeNameFormat.checkColumnName(colName);
}
public String getColName() {
return colName;
}
public String getComment() {
return comment;
}
@Override
public String toString() {
String str = "`" + colName + "`";
if (!comment.isEmpty()) {
str += " COMMENT \"" + comment + "\"";
}
return str;
}
}

17
fe/src/main/java/org/apache/doris/analysis/CreateTableStmt.java
View File

@ -17,6 +17,8 @@
package org.apache.doris.analysis;
import static org.apache.doris.catalog.AggregateType.BITMAP_UNION;
import org.apache.doris.catalog.AggregateType;
import org.apache.doris.catalog.Catalog;
import org.apache.doris.catalog.Column;
@ -48,8 +50,6 @@ import java.util.List;
import java.util.Map;
import java.util.Set;
import static org.apache.doris.catalog.AggregateType.BITMAP_UNION;
public class CreateTableStmt extends DdlStmt {
private static final Logger LOG = LogManager.getLogger(CreateTableStmt.class);
@ -65,6 +65,7 @@ public class CreateTableStmt extends DdlStmt {
private Map<String, String> properties;
private Map<String, String> extProperties;
private String engineName;
private String comment;
private static Set<String> engineNames;
@ -98,7 +99,8 @@ public class CreateTableStmt extends DdlStmt {
PartitionDesc partitionDesc,
DistributionDesc distributionDesc,
Map<String, String> properties,
Map<String, String> extProperties) {
Map<String, String> extProperties,
String comment) {
this.tableName = tableName;
if (columnDefinitions == null) {
this.columnDefs = Lists.newArrayList();
@ -118,6 +120,7 @@ public class CreateTableStmt extends DdlStmt {
this.extProperties = extProperties;
this.isExternal = isExternal;
this.ifNotExists = ifNotExists;
this.comment = Strings.nullToEmpty(comment);
this.tableSignature = -1;
}
@ -184,6 +187,10 @@ public class CreateTableStmt extends DdlStmt {
tableName = new TableName(tableName.getDb(), newTableName);
}
public String getComment() {
return comment;
}
@Override
public void analyze(Analyzer analyzer) throws AnalysisException, UserException {
super.analyze(analyzer);
@ -412,6 +419,10 @@ public class CreateTableStmt extends DdlStmt {
sb.append(")");
}
if (!Strings.isNullOrEmpty(comment)) {
sb.append("\nCOMMENT \"").append(comment).append("\"");
}
return sb.toString();
}

36
fe/src/main/java/org/apache/doris/analysis/CreateViewStmt.java
View File

@ -19,8 +19,8 @@ package org.apache.doris.analysis;
import org.apache.doris.catalog.Catalog;
import org.apache.doris.catalog.Column;
import org.apache.doris.catalog.ScalarType;
import org.apache.doris.catalog.PrimitiveType;
import org.apache.doris.catalog.ScalarType;
import org.apache.doris.common.AnalysisException;
import org.apache.doris.common.ErrorCode;
import org.apache.doris.common.ErrorReport;
@ -28,6 +28,7 @@ import org.apache.doris.common.UserException;
import org.apache.doris.mysql.privilege.PrivPredicate;
import org.apache.doris.qe.ConnectContext;
import com.google.common.base.Strings;
import com.google.common.collect.Lists;
import com.google.common.collect.Sets;
@ -36,13 +37,15 @@ import org.apache.logging.log4j.Logger;
import java.util.List;
import java.util.Set;
import java.util.stream.Collectors;
public class CreateViewStmt extends DdlStmt {
private static final Logger LOG = LogManager.getLogger(CreateViewStmt.class);
private final boolean ifNotExists;
private final TableName tableName;
private final List<String> columnNames;
private final List<ColWithComment> cols;
private final String comment;
private final QueryStmt viewDefStmt;
// Set during analyze
@ -52,10 +55,12 @@ public class CreateViewStmt extends DdlStmt {
private String inlineViewDef;
private QueryStmt cloneStmt;
public CreateViewStmt(boolean ifNotExists, TableName tableName, List<String> columnNames, QueryStmt queryStmt) {
public CreateViewStmt(boolean ifNotExists, TableName tableName, List<ColWithComment> cols,
String comment, QueryStmt queryStmt) {
this.ifNotExists = ifNotExists;
this.tableName = tableName;
this.columnNames = columnNames;
this.cols = cols;
this.comment = Strings.nullToEmpty(comment);
this.viewDefStmt = queryStmt;
finalCols = Lists.newArrayList();
}
@ -80,22 +85,26 @@ public class CreateViewStmt extends DdlStmt {
return inlineViewDef;
}
public String getComment() {
return comment;
}
/**
* Sets the originalViewDef and the expanded inlineViewDef based on viewDefStmt.
* If columnNames were given, checks that they do not contain duplicate column names
* and throws an exception if they do.
*/
private void createColumnAndViewDefs(Analyzer analyzer) throws AnalysisException, UserException {
if (columnNames != null) {
if (columnNames.size() != viewDefStmt.getColLabels().size()) {
if (cols != null) {
if (cols.size() != viewDefStmt.getColLabels().size()) {
ErrorReport.reportAnalysisException(ErrorCode.ERR_VIEW_WRONG_LIST);
}
// TODO(zc): type
for (int i = 0; i < columnNames.size(); ++i) {
for (int i = 0; i < cols.size(); ++i) {
PrimitiveType type = viewDefStmt.getBaseTblResultExprs().get(i).getType().getPrimitiveType();
finalCols.add(new Column(
columnNames.get(i),
ScalarType.createType(type)));
Column col = new Column(cols.get(i).getColName(), ScalarType.createType(type));
col.setComment(cols.get(i).getComment());
finalCols.add(col);
}
} else {
// TODO(zc): type
@ -117,13 +126,14 @@ public class CreateViewStmt extends DdlStmt {
// format view def string
originalViewDef = viewDefStmt.toSql();
if (columnNames == null) {
if (cols == null) {
inlineViewDef = originalViewDef;
return;
}
Analyzer tmpAnalyzer = new Analyzer(analyzer);
cloneStmt.substituteSelectList(tmpAnalyzer, columnNames);
List<String> colNames = cols.stream().map(c -> c.getColName()).collect(Collectors.toList());
cloneStmt.substituteSelectList(tmpAnalyzer, colNames);
inlineViewDef = cloneStmt.toSql();
// StringBuilder sb = new StringBuilder();
@ -146,7 +156,7 @@ public class CreateViewStmt extends DdlStmt {
@Override
public void analyze(Analyzer analyzer) throws AnalysisException, UserException {
if (columnNames != null) {
if (cols != null) {
cloneStmt = viewDefStmt.clone();
}
tableName.analyze(analyzer);

14
fe/src/main/java/org/apache/doris/backup/RestoreJob.java
View File

@ -502,11 +502,11 @@ public class RestoreJob extends AbstractJob {
if (localRange.equals(remoteRange)) {
// Same partition, same range
if (localRangePartInfo.getReplicationNum(localPartition.getId()) != restoreReplicationNum) {
status = new Status(ErrCode.COMMON_ERROR, "Parition " + backupPartInfo.name
status = new Status(ErrCode.COMMON_ERROR, "Partition " + backupPartInfo.name
+ " in table " + localTbl.getName()
+ " has different replication num '"
+ localRangePartInfo.getReplicationNum(localPartition.getId())
+ "' with parition in repository, which is " + restoreReplicationNum);
+ "' with partition in repository, which is " + restoreReplicationNum);
return;
}
genFileMapping(localOlapTbl, localPartition, tblInfo.id, backupPartInfo,
@ -516,19 +516,19 @@ public class RestoreJob extends AbstractJob {
backupPartInfo.versionHash));
} else {
// Same partition name, different range
status = new Status(ErrCode.COMMON_ERROR, "Parition " + backupPartInfo.name
status = new Status(ErrCode.COMMON_ERROR, "Partition " + backupPartInfo.name
+ " in table " + localTbl.getName()
+ " has different range with parition in repository");
+ " has different range with partition in repository");
return;
}
} else {
// If this is a single partitioned table.
if (localPartInfo.getReplicationNum(localPartition.getId()) != restoreReplicationNum) {
status = new Status(ErrCode.COMMON_ERROR, "Parition " + backupPartInfo.name
status = new Status(ErrCode.COMMON_ERROR, "Partition " + backupPartInfo.name
+ " in table " + localTbl.getName()
+ " has different replication num '"
+ localPartInfo.getReplicationNum(localPartition.getId())
+ "' with parition in repository, which is " + restoreReplicationNum);
+ "' with partition in repository, which is " + restoreReplicationNum);
return;
}
@ -549,7 +549,7 @@ public class RestoreJob extends AbstractJob {
= (RangePartitionInfo) remoteOlapTbl.getPartitionInfo();
Range<PartitionKey> remoteRange = remoteRangePartitionInfo.getRange(backupPartInfo.id);
if (!localRangePartitionInfo.checkRange(remoteRange)) {
status = new Status(ErrCode.COMMON_ERROR, "Parition " + backupPartInfo.name
status = new Status(ErrCode.COMMON_ERROR, "Partition " + backupPartInfo.name
+ " in table " + localTbl.getName()
+ " has conflict range with existing ranges");
return;

9
fe/src/main/java/org/apache/doris/catalog/BrokerTable.java
View File

@ -17,13 +17,15 @@
package org.apache.doris.catalog;
import com.google.common.collect.Maps;
import org.apache.commons.lang.StringEscapeUtils;
import org.apache.doris.common.DdlException;
import org.apache.doris.common.io.Text;
import org.apache.doris.thrift.TBrokerTable;
import org.apache.doris.thrift.TTableDescriptor;
import org.apache.doris.thrift.TTableType;
import com.google.common.collect.Maps;
import org.apache.commons.lang.StringEscapeUtils;
import org.apache.kudu.client.shaded.com.google.common.base.Strings;
import org.apache.kudu.client.shaded.com.google.common.collect.Lists;
import org.apache.logging.log4j.LogManager;
@ -56,7 +58,8 @@ public class BrokerTable extends Table {
super(TableType.BROKER);
}
public BrokerTable(long id, String name, List<Column> schema, Map<String, String> properties) throws DdlException {
public BrokerTable(long id, String name, List<Column> schema, Map<String, String> properties)
throws DdlException {
super(id, name, TableType.BROKER, schema);
validate(properties);
}

54
fe/src/main/java/org/apache/doris/catalog/Catalog.java
View File

@ -3374,7 +3374,9 @@ public class Catalog {
// create table
long tableId = Catalog.getInstance().getNextId();
OlapTable olapTable = new OlapTable(tableId, tableName, baseSchema, keysType, partitionInfo, distributionInfo);
OlapTable olapTable = new OlapTable(tableId, tableName, baseSchema, keysType, partitionInfo,
distributionInfo);
olapTable.setComment(stmt.getComment());
// set base index id
long baseIndexId = getNextId();
@ -3591,7 +3593,7 @@ public class Catalog {
long tableId = Catalog.getInstance().getNextId();
MysqlTable mysqlTable = new MysqlTable(tableId, tableName, columns, stmt.getProperties());
mysqlTable.setComment(stmt.getComment());
Table returnTable = null;
if (isRestore) {
returnTable = mysqlTable;
@ -3628,6 +3630,7 @@ public class Catalog {
long tableId = Catalog.getInstance().getNextId();
EsTable esTable = new EsTable(tableId, tableName, baseSchema, stmt.getProperties(), partitionInfo);
esTable.setComment(stmt.getComment());
if (!db.createTableWithLock(esTable, false, stmt.isSetIfNotExists())) {
ErrorReport.reportDdlException(ErrorCode.ERR_CANT_CREATE_TABLE, tableName, "table already exist");
@ -3728,6 +3731,7 @@ public class Catalog {
long tableId = Catalog.getInstance().getNextId();
BrokerTable brokerTable = new BrokerTable(tableId, tableName, columns, stmt.getProperties());
brokerTable.setComment(stmt.getComment());
brokerTable.setBrokerProperties(stmt.getExtProperties());
Table returnTable = null;
@ -3848,7 +3852,7 @@ public class Catalog {
sb.append(colocateTable).append("\"");
}
sb.append("\n);");
sb.append("\n)");
} else if (table.getType() == TableType.MYSQL) {
MysqlTable mysqlTable = (MysqlTable) table;
// properties
@ -3859,39 +3863,7 @@ public class Catalog {
sb.append("\"password\" = \"").append(hidePassword ? "" : mysqlTable.getPasswd()).append("\",\n");
sb.append("\"database\" = \"").append(mysqlTable.getMysqlDatabaseName()).append("\",\n");
sb.append("\"table\" = \"").append(mysqlTable.getMysqlTableName()).append("\"\n");
sb.append(");");
} else if (table.getType() == TableType.KUDU) {
KuduTable kuduTable = (KuduTable) table;
org.apache.kudu.client.KuduTable kTable = kuduTable.getKuduTable();
if (kTable == null) {
// real kudu table is not found
return;
}
// keys
sb.append("\n").append(KeysType.PRIMARY_KEYS.toSql()).append("(");
List<String> keysColumnNames = Lists.newArrayList();
for (Column column : kuduTable.getBaseSchema()) {
if (column.isKey()) {
keysColumnNames.add("`" + column.getName() + "`");
}
}
sb.append(Joiner.on(", ").join(keysColumnNames)).append(")");
// partition
KuduPartition rangePartition = kuduTable.getRangePartition();
if (rangePartition != null) {
sb.append("\n").append(rangePartition);
}
// distribution
KuduPartition hashPartition = kuduTable.getHashPartition();
sb.append("\n").append(hashPartition);
// properties
sb.append("\nPROPERTIES (\n");
sb.append("\"").append(PropertyAnalyzer.PROPERTIES_KUDU_MASTER_ADDRS).append("\" = \"");
sb.append(kuduTable.getMasterAddrs()).append("\")");
sb.append(")");
} else if (table.getType() == TableType.BROKER) {
BrokerTable brokerTable = (BrokerTable) table;
// properties
@ -3907,8 +3879,6 @@ public class Catalog {
hidePassword).toString());
sb.append("\n)");
}
sb.append(";");
} else if (table.getType() == TableType.ELASTICSEARCH) {
EsTable esTable = (EsTable) table;
@ -3936,9 +3906,14 @@ public class Catalog {
sb.append("\"index\" = \"").append(esTable.getIndexName()).append("\",\n");
sb.append("\"type\" = \"").append(esTable.getMappingType()).append("\",\n");
sb.append("\"transport\" = \"").append(esTable.getTransport()).append("\"\n");
sb.append(");");
sb.append(")");
}
if (!Strings.isNullOrEmpty(table.getComment())) {
sb.append("\nCOMMENT \"").append(table.getComment()).append("\"");
}
sb.append(";");
createTableStmt.add(sb.toString());
// 2. add partition
@ -5145,6 +5120,7 @@ public class Catalog {
long tableId = Catalog.getInstance().getNextId();
View newView = new View(tableId, tableName, columns);
newView.setComment(stmt.getComment());
newView.setInlineViewDef(stmt.getInlineViewDef());
newView.setOriginalViewDef(stmt.getInlineViewDef());
try {

4
fe/src/main/java/org/apache/doris/catalog/Column.java
View File

@ -186,6 +186,10 @@ public class Column implements Writable {
return this.stats;
}
public void setComment(String comment) {
this.comment = comment;
}
public String getComment() {
return comment;
}

26
fe/src/main/java/org/apache/doris/catalog/EsTable.java
View File

@ -17,6 +17,18 @@
package org.apache.doris.catalog;
import org.apache.doris.common.DdlException;
import org.apache.doris.common.io.Text;
import org.apache.doris.external.EsTableState;
import org.apache.doris.thrift.TEsTable;
import org.apache.doris.thrift.TTableDescriptor;
import org.apache.doris.thrift.TTableType;
import com.google.common.base.Strings;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import java.io.DataInput;
import java.io.DataOutput;
import java.io.IOException;
@ -25,17 +37,6 @@ import java.util.List;
import java.util.Map;
import java.util.zip.Adler32;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.doris.common.DdlException;
import org.apache.doris.common.io.Text;
import org.apache.doris.external.EsTableState;
import org.apache.doris.thrift.TEsTable;
import org.apache.doris.thrift.TTableDescriptor;
import org.apache.doris.thrift.TTableType;
import com.google.common.base.Strings;
public class EsTable extends Table {
private static final Logger LOG = LogManager.getLogger(EsTable.class);
@ -66,8 +67,7 @@ public class EsTable extends Table {
}
public EsTable(long id, String name, List<Column> schema,
Map<String, String> properties, PartitionInfo partitionInfo)
throws DdlException {
Map<String, String> properties, PartitionInfo partitionInfo) throws DdlException {
super(id, name, TableType.ELASTICSEARCH, schema);
this.partitionInfo = partitionInfo;
validate(properties);

4
fe/src/main/java/org/apache/doris/catalog/InlineView.java
View File

@ -17,10 +17,10 @@
package org.apache.doris.catalog;
import java.util.List;
import org.apache.doris.thrift.TTableDescriptor;
import java.util.List;
/**
* A fake catalog representation of an inline view. It's like a table. It has name
* and columns, but it won't have ids and it shouldn't be converted to Thrift.

2
fe/src/main/java/org/apache/doris/catalog/OlapTable.java
View File

@ -145,7 +145,7 @@ public class OlapTable extends Table {
}
public OlapTable(long id, String tableName, List<Column> baseSchema,
KeysType keysType, PartitionInfo partitionInfo, DistributionInfo defaultDistributionInfo) {
KeysType keysType, PartitionInfo partitionInfo, DistributionInfo defaultDistributionInfo) {
super(id, tableName, TableType.OLAP, baseSchema);
this.state = OlapTableState.NORMAL;

24
fe/src/main/java/org/apache/doris/catalog/Table.java
View File

@ -18,12 +18,14 @@
package org.apache.doris.catalog;
import org.apache.doris.analysis.CreateTableStmt;
import org.apache.doris.common.FeMetaVersion;
import org.apache.doris.common.UserException;
import org.apache.doris.common.io.Text;
import org.apache.doris.common.io.Writable;
import org.apache.doris.thrift.TTableDescriptor;
import com.google.common.base.Preconditions;
import com.google.common.base.Strings;
import com.google.common.collect.Lists;
import com.google.common.collect.Maps;
@ -72,6 +74,8 @@ public class Table extends MetaObject implements Writable {
// DO NOT persist this variable.
protected boolean isTypeRead = false;
// table(view)'s comment
protected String comment = "";
public Table(TableType type) {
this.type = type;
@ -193,6 +197,8 @@ public class Table extends MetaObject implements Writable {
for (Column column : fullSchema) {
column.write(out);
}
Text.writeString(out, comment);
}
@Override
@ -214,6 +220,12 @@ public class Table extends MetaObject implements Writable {
this.fullSchema.add(column);
this.nameToColumn.put(column.getName(), column);
}
if (Catalog.getCurrentCatalogJournalVersion() >= FeMetaVersion.VERSION_63) {
comment = Text.readString(in);
} else {
comment = "";
}
}
public boolean equals(Table table) {
@ -232,7 +244,7 @@ public class Table extends MetaObject implements Writable {
public String getEngine() {
if (this instanceof OlapTable) {
return "Palo";
return "Doris";
} else if (this instanceof MysqlTable) {
return "MySQL";
} else if (this instanceof SchemaTable) {
@ -250,10 +262,14 @@ public class Table extends MetaObject implements Writable {
}
public String getComment() {
if (this instanceof View) {
return "VIEW";
if (!Strings.isNullOrEmpty(comment)) {
return comment;
}
return "";
return type.name();
}
public void setComment(String comment) {
this.comment = Strings.nullToEmpty(comment);
}
public CreateTableStmt toCreateTableStmt(String dbName) {

1
fe/src/main/java/org/apache/doris/catalog/View.java
View File

@ -25,6 +25,7 @@ import org.apache.doris.common.UserException;
import org.apache.doris.common.io.Text;
import com.google.common.collect.Lists;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

2
fe/src/main/java/org/apache/doris/common/FeConstants.java
View File

@ -38,5 +38,5 @@ public class FeConstants {
// general model
// Current meta data version. Use this version to write journals and image
public static int meta_version = FeMetaVersion.VERSION_62;
public static int meta_version = FeMetaVersion.VERSION_63;
}

2
fe/src/main/java/org/apache/doris/common/FeMetaVersion.java
View File

@ -134,4 +134,6 @@ public final class FeMetaVersion {
public static final int VERSION_61 = 61;
// add param: doris_shuffle_partitions
public static final int VERSION_62 = 62;
// for table comment
public static final int VERSION_63 = 63;
}

1
fe/src/main/java/org/apache/doris/qe/SessionVariable.java
View File

@ -555,6 +555,5 @@ public class SessionVariable implements Serializable, Writable {
if (Catalog.getCurrentCatalogJournalVersion() >= FeMetaVersion.VERSION_62) {
exchangeInstanceParallel = in.readInt();
}
}
}

13
fe/src/test/java/org/apache/doris/analysis/CreateTableStmtTest.java
View File

@ -17,10 +17,9 @@
package org.apache.doris.analysis;
import org.apache.doris.catalog.Column;
import org.apache.doris.catalog.ScalarType;
import org.apache.doris.catalog.KeysType;
import org.apache.doris.catalog.PrimitiveType;
import org.apache.doris.catalog.ScalarType;
import org.apache.doris.common.AnalysisException;
import org.apache.doris.common.UserException;
import org.apache.doris.mysql.privilege.MockedAuth;
@ -97,7 +96,7 @@ public class CreateTableStmtTest {
public void testNormal() throws UserException, AnalysisException {
CreateTableStmt stmt = new CreateTableStmt(false, false, tblName, cols, "olap",
new KeysDesc(KeysType.AGG_KEYS, colsName), null,
new HashDistributionDesc(10, Lists.newArrayList("col1")), null, null);
new HashDistributionDesc(10, Lists.newArrayList("col1")), null, null, "");
stmt.analyze(analyzer);
Assert.assertEquals("testCluster:db1", stmt.getDbName());
Assert.assertEquals("table1", stmt.getTableName());
@ -108,7 +107,7 @@ public class CreateTableStmtTest {
public void testDefaultDbNormal() throws UserException, AnalysisException {
CreateTableStmt stmt = new CreateTableStmt(false, false, tblNameNoDb, cols, "olap",
new KeysDesc(KeysType.AGG_KEYS, colsName), null,
new HashDistributionDesc(10, Lists.newArrayList("col1")), null, null);
new HashDistributionDesc(10, Lists.newArrayList("col1")), null, null, "");
stmt.analyze(analyzer);
Assert.assertEquals("testDb", stmt.getDbName());
Assert.assertEquals("table1", stmt.getTableName());
@ -125,7 +124,7 @@ public class CreateTableStmtTest {
EasyMock.replay(analyzer);
CreateTableStmt stmt = new CreateTableStmt(false, false, tblNameNoDb, cols, "olap",
new KeysDesc(KeysType.AGG_KEYS, colsName), null,
new RandomDistributionDesc(10), null, null);
new RandomDistributionDesc(10), null, null, "");
stmt.analyze(analyzer);
}
@ -135,7 +134,7 @@ public class CreateTableStmtTest {
List<ColumnDef> emptyCols = Lists.newArrayList();
CreateTableStmt stmt = new CreateTableStmt(false, false, tblNameNoDb, emptyCols, "olap",
new KeysDesc(), null,
new RandomDistributionDesc(10), null, null);
new RandomDistributionDesc(10), null, null, "");
stmt.analyze(analyzer);
}
@ -144,7 +143,7 @@ public class CreateTableStmtTest {
// make defalut db return empty;
CreateTableStmt stmt = new CreateTableStmt(false, false, tblNameNoDb, invalidCols, "olap",
new KeysDesc(KeysType.AGG_KEYS, invalidColsName), null,
new RandomDistributionDesc(10), null, null);
new RandomDistributionDesc(10), null, null, "");
stmt.analyze(analyzer);
}
}

12
fe/src/test/java/org/apache/doris/catalog/ColocateTableTest.java
View File

@ -204,7 +204,7 @@ public class ColocateTableTest {
CreateTableStmt stmt = new CreateTableStmt(false, false, dbTableName1, columnDefs, "olap",
new KeysDesc(KeysType.AGG_KEYS, columnNames), null,
new HashDistributionDesc(numBucket, Lists.newArrayList("key1")), properties, null);
new HashDistributionDesc(numBucket, Lists.newArrayList("key1")), properties, null, "");
stmt.analyze(analyzer);
catalog.createTable(stmt);
}
@ -253,7 +253,7 @@ public class ColocateTableTest {
properties.put(PropertyAnalyzer.PROPERTIES_COLOCATE_WITH, groupName1);
CreateTableStmt secondStmt = new CreateTableStmt(false, false, dbTableName2, columnDefs, "olap",
new KeysDesc(KeysType.AGG_KEYS, columnNames), null,
new HashDistributionDesc(numBucket, Lists.newArrayList("key1")), properties, null);
new HashDistributionDesc(numBucket, Lists.newArrayList("key1")), properties, null, "");
secondStmt.analyze(analyzer);
catalog.createTable(secondStmt);
@ -308,7 +308,7 @@ public class ColocateTableTest {
int secondBucketNum = 2;
CreateTableStmt secondStmt = new CreateTableStmt(false, false, dbTableName2, columnDefs, "olap",
new KeysDesc(KeysType.AGG_KEYS, columnNames), null,
new HashDistributionDesc(secondBucketNum, Lists.newArrayList("key1")), properties, null);
new HashDistributionDesc(secondBucketNum, Lists.newArrayList("key1")), properties, null, "");
secondStmt.analyze(analyzer);
expectedEx.expect(DdlException.class);
@ -327,7 +327,7 @@ public class ColocateTableTest {
properties.put(PropertyAnalyzer.PROPERTIES_REPLICATION_NUM, "2");
CreateTableStmt secondStmt = new CreateTableStmt(false, false, dbTableName2, columnDefs, "olap",
new KeysDesc(KeysType.AGG_KEYS, columnNames), null,
new HashDistributionDesc(bucketNum, Lists.newArrayList("key1")), properties, null);
new HashDistributionDesc(bucketNum, Lists.newArrayList("key1")), properties, null, "");
secondStmt.analyze(analyzer);
expectedEx.expect(DdlException.class);
@ -344,7 +344,7 @@ public class ColocateTableTest {
properties.put(PropertyAnalyzer.PROPERTIES_COLOCATE_WITH, groupName1);
CreateTableStmt childStmt = new CreateTableStmt(false, false, dbTableName2, columnDefs, "olap",
new KeysDesc(KeysType.AGG_KEYS, columnNames), null,
new HashDistributionDesc(bucketNum, Lists.newArrayList("key1", "key2")), properties, null);
new HashDistributionDesc(bucketNum, Lists.newArrayList("key1", "key2")), properties, null, "");
childStmt.analyze(analyzer);
expectedEx.expect(DdlException.class);
@ -362,7 +362,7 @@ public class ColocateTableTest {
properties.put(PropertyAnalyzer.PROPERTIES_COLOCATE_WITH, groupName1);
CreateTableStmt childStmt = new CreateTableStmt(false, false, dbTableName2, columnDefs, "olap",
new KeysDesc(KeysType.AGG_KEYS, columnNames), null,
new HashDistributionDesc(bucketNum, Lists.newArrayList("key2")), properties, null);
new HashDistributionDesc(bucketNum, Lists.newArrayList("key2")), properties, null, "");
childStmt.analyze(analyzer);
expectedEx.expect(DdlException.class);

27
fe/src/test/java/org/apache/doris/catalog/CreateTableTest.java
View File

@ -17,11 +17,6 @@
package org.apache.doris.catalog;
import com.google.common.collect.Lists;
import mockit.Expectations;
import mockit.Injectable;
import mockit.Mock;
import mockit.MockUp;
import org.apache.doris.analysis.Analyzer;
import org.apache.doris.analysis.ColumnDef;
import org.apache.doris.analysis.CreateTableStmt;
@ -38,6 +33,9 @@ import org.apache.doris.persist.EditLog;
import org.apache.doris.qe.ConnectContext;
import org.apache.doris.system.SystemInfoService;
import org.apache.doris.task.AgentBatchTask;
import com.google.common.collect.Lists;
import org.junit.Before;
import org.junit.Rule;
import org.junit.Test;
@ -49,6 +47,11 @@ import java.util.Map;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
import mockit.Expectations;
import mockit.Injectable;
import mockit.Mock;
import mockit.MockUp;
public class CreateTableTest {
private TableName dbTableName;
@ -136,7 +139,7 @@ public class CreateTableTest {
CreateTableStmt stmt = new CreateTableStmt(false, false, dbTableName, columnDefs, "olap",
new KeysDesc(KeysType.AGG_KEYS, columnNames), null,
new HashDistributionDesc(1, Lists.newArrayList("key1")), null, null);
new HashDistributionDesc(1, Lists.newArrayList("key1")), null, null, "");
stmt.analyze(analyzer);
catalog.createTable(stmt);
@ -155,7 +158,7 @@ public class CreateTableTest {
CreateTableStmt stmt = new CreateTableStmt(false, false, dbTableName, columnDefs, "olap",
new KeysDesc(KeysType.AGG_KEYS, columnNames), null,
new HashDistributionDesc(1, Lists.newArrayList("key1")), null, null);
new HashDistributionDesc(1, Lists.newArrayList("key1")), null, null, "");
stmt.analyze(analyzer);
@ -191,7 +194,7 @@ public class CreateTableTest {
CreateTableStmt stmt = new CreateTableStmt(false, false, dbTableName, columnDefs, "olap",
new KeysDesc(KeysType.AGG_KEYS, columnNames), null,
new HashDistributionDesc(1, Lists.newArrayList("key1")), properties, null);
new HashDistributionDesc(1, Lists.newArrayList("key1")), properties, null, "");
stmt.analyze(analyzer);
expectedEx.expect(DdlException.class);
@ -229,7 +232,7 @@ public class CreateTableTest {
CreateTableStmt stmt = new CreateTableStmt(false, false, dbTableName, columnDefs, "olap",
new KeysDesc(KeysType.AGG_KEYS, columnNames), null,
new HashDistributionDesc(1, Lists.newArrayList("key1")), properties, null);
new HashDistributionDesc(1, Lists.newArrayList("key1")), properties, null, "");
stmt.analyze(analyzer);
expectedEx.expect(DdlException.class);
@ -262,7 +265,7 @@ public class CreateTableTest {
CreateTableStmt stmt = new CreateTableStmt(false, false, dbTableName, columnDefs, "olap",
new KeysDesc(KeysType.AGG_KEYS, columnNames), null,
new HashDistributionDesc(1, Lists.newArrayList("key1")), null, null);
new HashDistributionDesc(1, Lists.newArrayList("key1")), null, null, "");
stmt.analyze(analyzer);
expectedEx.expect(DdlException.class);
@ -299,7 +302,7 @@ public class CreateTableTest {
CreateTableStmt stmt = new CreateTableStmt(false, false, dbTableName, columnDefs, "olap",
new KeysDesc(KeysType.AGG_KEYS, columnNames), null,
new HashDistributionDesc(1, Lists.newArrayList("key1")), null, null);
new HashDistributionDesc(1, Lists.newArrayList("key1")), null, null, "");
stmt.analyze(analyzer);
expectedEx.expect(DdlException.class);
@ -332,7 +335,7 @@ public class CreateTableTest {
CreateTableStmt stmt = new CreateTableStmt(false, false, dbTableName, columnDefs, "olap",
new KeysDesc(KeysType.AGG_KEYS, columnNames), null,
new HashDistributionDesc(1, Lists.newArrayList("key1")), null, null);
new HashDistributionDesc(1, Lists.newArrayList("key1")), null, null, "");
stmt.analyze(analyzer);
expectedEx.expect(DdlException.class);