Go to file

ElvinWei 2d90f4b87c [feature-wip](statistics) step4: collect statistics by implementing statistics tasks (#8861 )

This pull request includes some implementations of the statistics(https://github.com/apache/incubator-doris/issues/6370), it will not affect any existing code and users will not be able to create statistics job.

Now only MetaStatisticsTask that directly collects statistics by reading FE meta is implemented. SQLStatisticsTask is still being implemented, it needs to query BE through FE.

The following is the function implemented by this pr: 
1. Support statistics collection for partitioned and non-partitioned tables. For partitioned tables, the collection of statistics for the specified partition is implemented.
2. When the task is divided, it is divided according to the partition table and the non-partition table. The most fine-grained is to the tablet level. A matetask collects as many statistics as possible. 
3. Add partition statistics (Table -> Partition -> Column). For example, the size of the table, the number of rows, the size of the partition, the number of rows, the maximum and minimum values of the columns, etc.
4. Display and modify partition-level statistics.
 …

2022-07-19 16:22:25 +08:00

.github

[action](Nereids): add label auto for nereids UT. (#10665 )

2022-07-07 18:21:04 +08:00

[Enhancement] [Memory] Limit memory usage use process actual physical memory (#10924 )

2022-07-19 11:08:39 +08:00

bin

release memory allocated in agg function in vec stream load (#10739 )

2022-07-16 15:32:53 +08:00

build-support

support different python version in build-support scripts (#10378 )

2022-07-11 23:23:45 +08:00

conf

[enhancement](be) be asan add asan_suppr.conf to ignore known leak. (#10768 )

2022-07-12 19:51:34 +08:00

contrib/udf

[UDF] support RPC udaf part 1: support create RPC udaf in fe (#8510 )

2022-04-21 17:38:58 +08:00

dist

[TLP](step-1) Remove incubator prefix (#10230 )

2022-06-19 19:34:52 +08:00

docker

[TLP](step-1) Remove incubator prefix (#10230 )

2022-06-19 19:34:52 +08:00

docs

[docs] Changing the symbol of dataX doriswriter table creation statement (#10632 )

2022-07-19 10:15:27 +08:00

extension

[compile]Update init-env.sh (#10451 )

2022-06-30 11:28:06 +08:00

[feature-wip](statistics) step4: collect statistics by implementing statistics tasks (#8861 )

2022-07-19 16:22:25 +08:00

fe_plugins

[TLP](step-1) Remove incubator prefix (#10230 )

2022-06-19 19:34:52 +08:00

fs_brokers/apache_hdfs_broker

[TLP](step-1) Remove incubator prefix (#10230 )

2022-06-19 19:34:52 +08:00

gensrc

[feature-wip](multi-catalog) Optimize threads and thrift interface of FileScanNode (#10942 )

2022-07-18 20:50:34 +08:00

regression-test

Revert "[regression] Add ssb sf1 test under unique table with zstd (#10957 )" (#10992 )

2022-07-19 15:44:32 +08:00

samples

[feature](udf) Vectorization support remote udaf #10683 (#10685 )

2022-07-18 17:15:34 +08:00

thirdparty

[enhancement](thirdparty) Support building thirdparty on macOS (#10677 )

2022-07-18 10:50:30 +08:00

tools

change date to int, good for performance (#10889 )

2022-07-16 22:35:43 +08:00

[improvement]Remove the page button on the System page (#10900 )

2022-07-16 06:00:08 +08:00

webroot

[License] Organize and modify the license of the code (#4371 )

2020-08-24 21:51:55 +08:00

.asf.yaml

[TLP](step-2) update .asf.yaml (#10273 )

2022-06-21 09:43:56 +08:00

.clang-format

[refactor][style] Use clang-format to sort includes (#9483 )

2022-05-10 21:25:35 +08:00

.clang-format-ignore

[chore](clang-format)(license-eye) Add Clang Format/Skywalking eyes github action (#7132 )

2021-11-24 10:41:02 +08:00

.clang-tidy

[refactor] add some clang-tidy checks && some code style fix (#8752 )

2022-03-31 13:53:41 +08:00

.clangd

[enhancement][diagnostics] Add a diagnostic: detect unused includes (#9117 )

2022-06-08 11:52:48 +08:00

.editorconfig

[enhancement](thirdparty) Support building thirdparty on macOS (#10677 )

2022-07-18 10:50:30 +08:00

.gitattributes

Enable auto convert when check in (#1926 )

2019-10-09 22:31:27 +08:00

.gitignore

[Chore][Compile] fix compile fail on clang (#10837 )

2022-07-18 19:21:01 +08:00

.gitmodules

[chore]replace checkstyle action with mvn checkstyle:check (#10474 )

2022-06-30 11:20:50 +08:00

.licenserc.yaml

[website](doc)add package-lock.json to resolve docs build failure (#10558 )

2022-07-02 17:20:11 +08:00

.rat-excludes

[improvement] Improve sig handler (#8545 )

2022-03-22 10:40:31 +08:00

build_plugin.sh

[Fix Bug] Fix ehco command not found (#9021 )

2022-04-15 13:43:47 +08:00

build.sh

[Enhancement] [Memory] add strict memory usage compile option STRICT_MEMORY_USE (#10936 )

2022-07-18 16:16:43 +08:00

CODE_OF_CONDUCT.md

[Doc] Create CODE_OF_CONDUCT.md (#4070 )

2020-07-14 22:28:38 +08:00

CONTRIBUTING_CN.md

[TLP](step-1) Remove incubator prefix (#10230 )

2022-06-19 19:34:52 +08:00

CONTRIBUTING.md

[TLP](step-1) Remove incubator prefix (#10230 )

2022-06-19 19:34:52 +08:00

env.sh

[refactor] fix warings when compile with clang (#8069 )

2022-02-19 11:29:02 +08:00

LICENSE.txt

[website] add website external resource (#10416 )

2022-06-26 01:22:14 +08:00

NOTICE.txt

[TLP](step-1) Remove incubator prefix (#10230 )

2022-06-19 19:34:52 +08:00

README.md

[Bug][docs] Fix wrong links in README.md (#10394 )

2022-07-04 14:44:23 +08:00

run-be-ut.sh

[Enhancement] [Memory] add strict memory usage compile option STRICT_MEMORY_USE (#10936 )

2022-07-18 16:16:43 +08:00

run-fe-ut.sh

[Fix Bug] Fix ehco command not found (#9021 )

2022-04-15 13:43:47 +08:00

run-regression-test.sh

[fix](regression-test) fix run-regression-test Xmx2048m param (#10234 )

2022-06-17 23:30:44 +08:00

tsan_suppressions

[TSAN] Fix tsan bugs (part 1) (#5162 )

2021-01-15 09:45:11 +08:00

README.md

Apache Doris

Doris is an MPP-based interactive SQL data warehousing for reporting and analysis. Its original name was Palo, developed in Baidu. After donated to Apache Software Foundation, it was renamed Doris.

Doris provides high concurrent low latency point query performance, as well as high throughput queries of ad-hoc analysis.
Doris provides batch data loading and real-time mini-batch data loading.
Doris provides high availability, reliability, fault tolerance, and scalability.

The main advantages of Doris are the simplicity (of developing, deploying and using) and meeting many data serving requirements in a single system. For details, refer to Overview.

Official website: https://doris.apache.org/

License

Apache License, Version 2.0

Note

Some licenses of the third-party dependencies are not compatible with Apache 2.0 License. So you need to disable some Doris features to be complied with Apache 2.0 License. For details, refer to the thirdparty/LICENSE.txt

Technology

Doris mainly integrates the technology of Google Mesa and Apache Impala, and it is based on a column-oriented storage engine and can communicate by MySQL client.