Commit Graph

642 Commits

Author SHA1 Message Date
c8899ee5bd [Build][ARM] Fix some compilation problems on ARM64 (#6076)
1. Disable libhdfs3 on ARM, because it doesn't support ARM now.
2. Add compilation doc for ARM64
2021-06-23 09:38:16 +08:00
72d1a3b39c fix spring boot web maximum upload file limit config (#6070)
Co-authored-by: zouxinyi <zouxinyi@baidu.com>
2021-06-22 10:46:26 +08:00
b9ad34736d [Feature] Support recording custom number of backup and restore task information (#5947)
* Record all backup jobs and support where clause
2021-06-22 09:19:54 +08:00
4fe8bdfe1d [Doc] Update install-deploy.md (#5968)
improve doc to avoid error of installing be in hadoop cluster
2021-06-20 16:09:13 +08:00
0ddd5da926 [DOC]Organize FE configuration file description (#5975)
* Organize FE configuration file description

Organize FE configuration file description

* Delete redundant numbers

Delete redundant numbers

* Add two configuration parameters of spring boot upload file

Add two configuration parameters of spring boot upload file

* Add configuration instructions

Add configuration instructions

* Fix typos

Fix typos

* Add English documentation of BE configuration

Add English documentation of BE configuration

* Modify style

Modify style

* Modify punctuation

Modify punctuation

* Correct the errors in the text

Correct the errors in the text

* Modify some ads and content issues

Modify some ads and content issues
2021-06-18 09:22:29 +08:00
0145bdb1f0 [Doc] Fix a typo (#6025)
Fix a typo in udaf-orthogonal-bitmap-manual.md
2021-06-16 09:41:32 +08:00
800c2c41bd [Docs] update data-model-rollup.md create table ddl (#6014)
update data-model-rollup.md create table ddl
2021-06-16 09:40:38 +08:00
d33a6d1b98 [Function] Support date function: yearweek(), week(), makedate(). (#6000) 2021-06-10 17:38:25 +08:00
d9c128b744 [BrokerLoad] Support read properties for broker load when read data (#5845)
* [BrokerLoad] support read properties for broker load when read data

Co-authored-by: caiconghui <caiconghui@xiaomi.com>
2021-06-09 14:59:55 +08:00
2cc5f776ce Corrected a typo in the Chinese document (#5978) 2021-06-08 16:44:25 +08:00
af34f877e9 Update zh-CN ALTER TABLE.md (#5982)
Add distribution column cannot be modified
2021-06-08 16:44:13 +08:00
8ea0ae4792 [Docker] update docker hub url to official repo (#5972) 2021-06-06 22:08:21 +08:00
9ade3bbd8a Add Doris error code table (#5935) 2021-06-02 09:59:41 +08:00
8956641296 [Docs] update PMC Members & Committers (#5941)
* update PMC Members & Committers
2021-06-01 18:11:34 +08:00
c844e602a7 [BUG] Fix the bug of Desc Query cause Unknown error and some doc revise (#5921) 2021-05-28 11:21:31 +08:00
a29dd42b47 [BUG][Document] Fix the bug that failed to build the help module (#5917)
There are multiple entries with same key in help documents, which will cause build help module failed.
2021-05-27 22:07:15 +08:00
3cb0174aa2 [Doc] Fix demo in 'CREATE TABLE' (#5865) 2021-05-27 22:03:51 +08:00
cbe1b48b71 [Docs] Fix typo in docs zh-CN CREATE TABLE.md (#5864) 2021-05-27 22:03:26 +08:00
832faa4377 [Docs] Fix wrong link (#5911) 2021-05-27 10:18:52 +08:00
a415134336 [Release] Release 0.14 and update 0.14 download url (#5906)
* update 0.14 download url
2021-05-26 13:55:40 +08:00
fc1389240f [Doc]Modify the flink connector document's instructions on enabling http v2 (#5883)
[doc]Modify the flink connector document's instructions on enabling http v2
2021-05-26 10:01:31 +08:00
510606ddd4 [DynamicPartition] Support specifying hot data partition (#5877)
In some scenarios, when users use dynamic partitions, they hope to use Doris' hierarchical storage
function at the same time.
For example, for the dynamic partition rule of partitioning by day, we hope that the partitions of the last 3 days
are stored on the SSD storage medium and automatically migrated to the HDD storage medium after expiration.

This CL add a new dynamic partition property: "hot_partition_num".
This parameter is used to specify how many recent partitions need to be stored on the SSD storage medium.
2021-05-26 10:00:24 +08:00
ba69f7a7c8 [Command] [SQL] Add show database/table/partition id command (#5807)
In BE, when a problem happened, in the log, we can find the database id, table id, partition id,
but no database name, table name, partition name.

In FE, there also no way to find database name/table name/partition name accourding to
database id/table id/partition id. Therefore, this patch add 3 new commands:

1. show database id;
mysql> show database 10002;
+----------------------+
| DbName               |
+----------------------+
| default_cluster:test |
+----------------------+

2. show table id;
mysql> show table 11100;
+----------------------+-----------+-------+
| DbName               | TableName | DbId  |
+----------------------+-----------+-------+
| default_cluster:test | table2    | 10002 |
+----------------------+-----------+-------+

3. show partition id;
mysql> show partition 11099;
+----------------------+-----------+---------------+-------+---------+
| DbName               | TableName | PartitionName | DbId  | TableId |
+----------------------+-----------+---------------+-------+---------+
| default_cluster:test | table2    | p201708       | 10002 | 11100   |
+----------------------+-----------+---------------+-------+---------+
2021-05-26 09:58:02 +08:00
a55b12da90 [Doc] Fix some flaws in hll-type document (#5795)
Fix some flaws in hll-type document.
2021-05-26 09:56:55 +08:00
76eca9de56 [Bug] Kill the FE process when writing BDBJE journal failed (#5861)
1. When an oom error occurs when writing bdbje, catch the error and exit the process.
2. Increase the timeout period of bdbje replica ack and change it to a configuration.
2021-05-22 23:38:47 +08:00
07ad038870 [Feature][RoutineLoad] Support for consuming kafka from the point of time (#5832)
Support when creating a kafka routine load, start consumption from a specified point in time instead of a specific offset.
eg:
```
FROM KAFKA
(
    "kafka_broker_list" = "broker1:9092,broker2:9092",
    "kafka_topic" = "my_topic",
    "property.kafka_default_offsets" = "2021-10-10 11:00:00"
);

or

FROM KAFKA
(
    "kafka_broker_list" = "broker1:9092,broker2:9092",
    "kafka_topic" = "my_topic",
    "kafka_partitions" = "0,1,2",
    "kafka_offsets" = "2021-10-10 11:00:00, 2021-10-10 11:00:00, 2021-10-10 12:00:00"
);
```

This PR also reconstructed the analysis method of properties when creating or altering
routine load jobs, and unified the analysis process in the `RoutineLoadDataSourceProperties` class.
2021-05-22 23:37:53 +08:00
12e4ff2689 [Doc] Fix doc for 'SHOW EXPORT' (#5840) 2021-05-19 09:31:57 +08:00
3406f09f55 [Doc] Fix metadata-design.md misspelling (#5837)
整体架构: "负责数据存储于管理"-> "负责数据存储与管理"
2021-05-19 09:31:06 +08:00
9eacd0a89c [Doc] remove storage_type from docs (#5814) 2021-05-19 09:29:15 +08:00
bf4443578c [Feature] Support show view statement for table (#5813)
Help to find all views which contain the given table
2021-05-19 09:29:00 +08:00
1a81b9e160 [MemTracker] Some enchance of MemTracker (#5783)
1 Make some MemTracker have reasonable parent MemTracker not the root tracker
2 Make each MemTracker can be easily to trace.
3 Add show level of MemTracker to reduce the MemTracker show in the web page to have a way to control show how many tracker in web page.
2021-05-19 09:27:50 +08:00
65ff464e3d [Feature] Support show data order by (#5770)
Currently, the `show data` does not support sorting. When the number of tables increases, it is inconvenient to manage. Need to support sorting

like:
```
mysql>  show data order by ReplicaCount desc,Size asc;
+-----------+-------------+--------------+
| TableName | Size        | ReplicaCount |
+-----------+-------------+--------------+
| table_c   | 3.102 KB    | 40           |
| table_d   | .000        | 20           |
| table_b   | 324.000 B   | 20           |
| table_a   | 1.266 KB    | 10           |
| Total     | 4.684 KB    | 90           |
| Quota     | 1024.000 GB | 1073741824   |
| Left      | 1024.000 GB | 1073741734   |
+-----------+-------------+--------------+
```
2021-05-19 09:27:27 +08:00
add8c4bb74 [Load] Support reading multi-line json objects for JsonScanner (#5774)
Co-authored-by: caiconghui <caiconghui@xiaomi.com>
2021-05-18 15:44:45 +08:00
9431090c0e Add flink doris connector design document (#5709)
* Solve the situation that the hardware information of the Web UI home page cannot be loaded

Solve the situation that the hardware information of the Web UI home page cannot be loaded

* Add flink doris connector design document

Add flink doris connector design document

* flink doris connector design english docment

flink doris connector design english docment

Co-authored-by: zhangjf@shuhaisc.com <zhangfeng800729>
2021-05-17 11:53:43 +08:00
d784cc06f6 [Doc] Flink doris connector document modification (#5769) 2021-05-12 10:59:35 +08:00
d7d50f7ffa [Optimize] Speed up the bulk data load to ODBC table. (#5765)
1. Batch Insert
2. Use fmt to repalce stringstream
3. Add some profile of ODBC_TABLE_SINK
2021-05-12 10:58:52 +08:00
11cce06962 [Feature] Support create history dynamic partition (#5703)
1. Add a new dynamic partition property `create_history_partition`.
    If set to true, Doris will create all partitions from `start` to `end`.

2. Add a new FE config `max_dynamic_partition_num`
    To limit the number of partitions created when creating one table.
2021-05-08 12:05:19 +08:00
3fdfe0ba6f [Bug-fix] Export specified column (#5759)
The code logic error causes the user to specify the export column, which may not be effective.
The PR fix this problem.
2021-05-08 10:56:45 +08:00
8850cfe2ad [Compaction] Modify compaction logic (#5737)
1. Add /api/compaction/run_status to show the running compaction tasks.
2. Support do base and cumulative compaction for one tablet at same time.
3. Modify some log level.
4. Add a feedback document.
2021-05-07 11:18:47 +08:00
6a1313594c [Thirdparty] Fix thirdparty build failed (#5754) 2021-05-05 10:20:25 +08:00
5bfae7674c Fix cancel load path variable LABEL_KEY (#5724) (#5725)
Co-authored-by: 迟成 <chicheng@meituan.com>
2021-04-30 10:14:51 +08:00
507664a44d fix spelling mistake in alter table doc (#5723)
Co-authored-by: weizuo <weizuo@xiaomi.com>
2021-04-30 10:12:50 +08:00
8b7ae1784a [Apache] Change download link to archive page (#5731)
* [Apache] Change download link to archive page

Only the latest package should appear in svn.
The old release package already has been archived, so the download link should be replaced by archive page.
2021-04-29 15:49:34 +08:00
9001fd28f4 support show stream load sql (#5488)
Co-authored-by: weizuo <weizuo@xiaomi.com>
2021-04-29 09:20:35 +08:00
c7af83b7c7 Support specify certain columns in export stmt (#5689)
Data export function, add certain columns that can be exported to the table.
Export stmt properties ("columns" = "k1, k2, k3");
2021-04-27 18:35:58 +08:00
de87f4ae84 [Feature] Add list partition support (#5529)
Add list partition support
2021-04-24 17:42:27 +08:00
a1fa392f1f [Doc] Missing sidebar of flink connector doc (#5695) 2021-04-23 22:21:06 +08:00
7eea811f6b [Feature] Flink Doris Connector (#5372) (#5375) 2021-04-23 09:43:48 +08:00
2b7d7e3385 fix typo (#5652) 2021-04-20 09:13:56 +08:00
9403157da4 [DOC] Add docs of warning of Docker env 1.3 of JDK 11 and some doc of FE config. (#5628) 2021-04-14 00:10:30 +08:00