0f14408f13
Related issue: #2663, #2828. This CL support loading data into specified temporary partitions. ``` INSERT INTO tbl TEMPORARY PARTITIONS(tp1, tp2, ..) ....; curl .... -H "temporary_partition: tp1, tp, .. " .... LOAD LABEL db1.label1 ( DATA INFILE("xxxx") INTO TABLE `tbl2` TEMPORARY PARTITION(tp1, tp2, ...) ... ``` NOTICE: this CL change the FE meta version to 77. There 3 major changes in this CL ## Syntax reorganization Reorganized the syntax related to the `specify-partitions`. Removed some redundant syntax definitions, and unified the syntax related to the `specify-partitions` under one syntax entry. ## Meta refactor In order to be able to support specifying temporary partitions, I made some changes to the way the partition information in the table is stored. Partition information is now organized as follows: The following two maps are reserved in OlapTable for storing formal partitions: ``` idToPartition nameToPartition ``` Use the `TempPartitions` class for storing temporary partitions. All the partition attributes of the formal partition and the temporary partition, such as the range, the number of replicas, and the storage medium, are all stored in the `partitionInfo` of the OlapTable. In `partitionInfo`, we use two maps to store the range of formal partition and temporary partition: ``` idToRange idToTempRange ``` Use separate map is because the partition ranges of the formal partition and the temporary partition may overlap. Separate map can more easily check the partition range. All partition attributes except the partition range are stored using the same map, and the partition id is used as the map key. ## Method to get partition A table may contain both formal and temporary partitions. There are several methods to get the partition of a table. Typically divided into two categories: 1. Get partition by id 2. Get partition by name According to different requirements, the caller may want to obtain a formal partition or a temporary partition. These methods are described below in order to obtain the partition by using the correct method. 1. Get by name This type of request usually comes from a user with partition names. Such as `select * from tbl partition(p1);`. This type of request has clear information to indicate whether to obtain a formal or temporary partition. Therefore, we need to get the partition through this method: `getPartition(String partitionName, boolean isTemp)` To avoid modifying too much code, we leave the `getPartition(String partitionName)`, which is same as: `getPartition(partitionName, false)` 2. Get by id This type of request usually means that the previous step has obtained certain partition ids in some way, so we only need to get the corresponding partition through this method: `getPartition(long partitionId)`. This method will try to get both formal partitions and temporary partitions. 3. Get all partition instances Depending on the requirements, the caller may want to obtain all formal partitions, all temporary partitions, or all partitions. Therefore we provide 3 methods, the caller chooses according to needs. `getPartitions()` `getTempPartitions()` `getAllPartitions()`
Philosophy
write once, use everywhere
Documentations will be written once, and will be converted to other format according to different application scenarios.
Implementation
+---------------+
| Documentation |
+-------+-------+
|
+-------+-------+
| Doc Builder |
+-------+-------+
|
+--------------------------------+
| | |
+---+---+ +---+----+ +-----+----+
| PDF | | HTML | .... | Help Doc |
+-------+ +--------+ +----------+
Documentation：Text contents which is written by human. And this is the only place for documentation. Doc Builder: Tools that convert documentations to other format, such as PDF, HTML. There could be many tools, and we can use different tools to convert documentation to different formats.
Organization
docs/documentation: Root directory for documentation. And for different languages, there is a root directory for it. For example,docs/documentation/cnis the Chinese documentation's root directory.docs/scripts: Place ofDoc Builder.docs/resources: Resources that are referenced in documentation, such as pictures.docs/website: A website for documentations built with Sphinx using a theme provided by Read-the-Docs.
Constraints
- All documents are written in Markdown format, and file name is end with ".md".
- All documents are started with level 1 title
# Title, and should have only one level 1 title. - Names of file and directory are in lowercase letters, and use dashes as separator.
- Documentation can be constructed as a directory or a single Markdown file, these two formats equal with each other in logical. Relationship is represented by parent-child directory in directory format, and by title level in file format. It is recommended to use directory format to manage a large documentation, because it is easy to maintain.
- A directory corresponds to a title, and readme.md in this directory is its content. Other documents in this directory is its sub-sections.
- For manual like section, such as function description, there should be
Description,Syntax,Examplessection in documents.
Level Directories
- doris-concepts
- installing
- getting-started
- administrator-guide
- sql-references
- best-practices
- internals
- community
Each directory, or its sub directories should contain a file index.rst, for constructing the navibar of the website. For example:
documentation/
└── cn
├── administrator-guide
│ ├── index.rst
│ ├── http-actions
│ │ └── index.rst
│ ├── load-data
│ │ ├── index.rst
│ ├── operation
│ │ ├── index.rst
├── extending-doris
│ ├── index.rst
└── sql-reference
├── index.rst
│ ├── date-time-functions
│ │ ├── index.rst
Docs Styles
There are some styles need to be followed.
SQL-Statement
Docs under documentation/cn/sql-reference/sql-statements/ must obey the following style
# TITLE(capital)
## description
The description of this doc. The "## description" must be reserved, with a following empty line.
## keyword
The keyword of this doc. Usually, this can be the title of this doc.
The "## keyword" must be reserved, with a following empty line.