The current compaction selection strategy and cumulative point update logic will cause the cumulative compaction to not work, and all compaction tasks will be completed only by the base compaction. This can cause a large number of data versions to pile up. In the current cumulative point update logic, when a cumulative cannot select enough number of rowsets, it will directly increase the cumulative point. Therefore, when the data version generates the same speed as the cumulative compaction polling, it will cause the cumulative point to continuously increase without triggering the cumulative compaction. The new strategy mainly modifies the update logic of cumulative point to ensure that the above problems do not occur. At the same time, the new strategy also takes into account the problem that compaction cannot be performed if cumulative points stagnate for a long time. Cumulative points will be forced to increase through threshold settings to ensure that compaction has a chance to execute. Also add a new HTTP API to view the compaction status of specified tablet. See `compaction-action.md` for details.
Philosophy
write once, use everywhere
Documentations will be written once, and will be converted to other format according to different application scenarios.
Implementation
+---------------+
| Documentation |
+-------+-------+
|
+-------+-------+
| Doc Builder |
+-------+-------+
|
+--------------------------------+
| | |
+---+---+ +---+----+ +-----+----+
| PDF | | HTML | .... | Help Doc |
+-------+ +--------+ +----------+
Documentation:Text contents which is written by human. And this is the only place for documentation. Doc Builder: Tools that convert documentations to other format, such as PDF, HTML. There could be many tools, and we can use different tools to convert documentation to different formats.
Organization
docs/documentation: Root directory for documentation. And for different languages, there is a root directory for it. For example,docs/documentation/cnis the Chinese documentation's root directory.docs/scripts: Place ofDoc Builder.docs/resources: Resources that are referenced in documentation, such as pictures.docs/website: A website for documentations built with Sphinx using a theme provided by Read-the-Docs.
Constraints
- All documents are written in Markdown format, and file name is end with ".md".
- All documents are started with level 1 title
# Title, and should have only one level 1 title. - Names of file and directory are in lowercase letters, and use dashes as separator.
- Documentation can be constructed as a directory or a single Markdown file, these two formats equal with each other in logical. Relationship is represented by parent-child directory in directory format, and by title level in file format. It is recommended to use directory format to manage a large documentation, because it is easy to maintain.
- A directory corresponds to a title, and readme.md in this directory is its content. Other documents in this directory is its sub-sections.
- For manual like section, such as function description, there should be
Description,Syntax,Examplessection in documents.
Level Directories
- doris-concepts
- installing
- getting-started
- administrator-guide
- sql-references
- best-practices
- internals
- community
Each directory, or its sub directories should contain a file index.rst, for constructing the navibar of the website. For example:
documentation/
└── cn
├── administrator-guide
│ ├── index.rst
│ ├── http-actions
│ │ └── index.rst
│ ├── load-data
│ │ ├── index.rst
│ ├── operation
│ │ ├── index.rst
├── extending-doris
│ ├── index.rst
└── sql-reference
├── index.rst
│ ├── date-time-functions
│ │ ├── index.rst
Docs Styles
There are some styles need to be followed.
SQL-Statement
Docs under documentation/cn/sql-reference/sql-statements/ must obey the following style
# TITLE(capital)
## description
The description of this doc. The "## description" must be reserved, with a following empty line.
## keyword
The keyword of this doc. Usually, this can be the title of this doc.
The "## keyword" must be reserved, with a following empty line.