doris

Files

zhbinbin f92428248f Support udaf_orthogonal_bitmap (#4198 )

The original Doris bitmap aggregation function has poor performance on the intersection and union set of bitmap cardinality of more than one billion. There are two reasons for this. The first is that when the bitmap cardinality is large, if the data size exceeds 1g, the network / disk IO time consumption will increase; The second point is that all the sink data of the back-end be instance are transferred to the top node for intersection and union calculation, which leads to the pressure on the top single node and becomes the bottleneck.

My solution is to create a fixed schema table based on the Doris fragmentation rule, and hash fragment the ID range based on the bitmap, that is, cut the ID range vertically to form a small cube. Such bitmap blocks will become smaller and evenly distributed on all back-end be instances. Based on the schema table, some new high-performance udaf aggregation functions are developed. All Scan nodes participate in intersection and union calculation, and top nodes only summarize

The design goal is that the base number of bitmap is more than 10 billion, and the response time of cross union set calculation of 100 dimensional granularity is within 5 s.

There are three udaf functions in this commit: orthogonal_bitmap_intersect_count, orthogonal_bitmap_union_count, orthogonal_bitmap_intersect.

2020-08-19 10:29:13 +08:00

administrator-guide

[Compaction]Compaction rules optimization (#4212 )

2020-08-19 09:34:13 +08:00

community

[website] modify download links & remove some links' suffix _EN(master) (#3573 )

2020-05-15 14:03:28 +08:00

developer-guide

Generate jave files using maven (#4133 )

2020-08-05 15:20:39 +08:00

downloads

[website] modify download links & remove some links' suffix _EN(master) (#3573 )

2020-05-15 14:03:28 +08:00

extending-doris

Support udaf_orthogonal_bitmap (#4198 )

2020-08-19 10:29:13 +08:00

getting-started

[SQL][Function] Add approx_count_distinct() function (#4221 )

2020-08-01 17:54:19 +08:00

installing

[Doc] Update compilation.md (#4297 )

2020-08-09 20:50:33 +08:00

internal

Update support batch delete storage design document (#4234 )

2020-08-18 15:37:14 +08:00

sql-reference

[Doc] modify the document error (#4357 )

2020-08-17 23:06:23 +08:00

README.md

[Website] Redesign the home page of document website (master) (#4069 )

2020-07-16 11:36:24 +08:00

README.md

home, heroImage, heroBgImage, heroText, tagline, structure, features, cases, actionText, actionLink

home

heroImage

heroBgImage

heroText

tagline

structure

features

cases

actionText

actionLink

true

/images/home/banner-stats.png

/images/home/hero-bg.png

Welcome to

Apache Doris

A fast MPP database for all modern analytics on big data.

title

subTitle

descriptions

image

actionText

actionLink

Apache Doris

Apache Doris is a modern MPP analytical database product. It can provide sub-second queries and efficient real-time data analysis. With it's distributed architecture, up to 10PB level datasets will be well supported and easy to operate.

Apache Doris can meet various data analysis demands, including history data reports, real-time data analysis, interactive data analysis, and exploratory data analysis. Make your data analysis easier!

/images/home/structure-fresh.png

Learn More

/en/getting-started/basic-usage

title

subTitle

list

Apache Doris Core Features

title	icon
Modern MPP architecture	/images/home/struct.png

title	icon
Getting result of a query within one second	/images/home/clock.png

title	icon
Support standard SQL language, compatible with MySQL protocol	/images/home/sql.png

title	icon
Vectorized SQL executor	/images/home/program.png

title	icon
Effective data model for aggregation	/images/home/aggr.png

title	icon
Rollup，novel pre-computation mechanism	/images/home/rollup.png

title	icon
High performance, high availability, high reliability	/images/home/cpu.png

title	icon
easy for operation，Elastic data warehouse for big data	/images/home/dev.png

title

subTitle

list

Apache Doris Users

logo	alt
/images/home/logo-meituan.png	美团

logo	alt
/images/home/logo-xiaomi.png	小米

logo	alt
/images/home/logo-jd.png	京东

logo	alt
/images/home/logo-huawei.png	华为

logo	alt
/images/home/logo-baidu.png	百度

logo	alt
/images/home/logo-weibo.png	新浪微博

logo	alt
/images/home/logo-zuoyebang.png	作业帮

logo	alt
/images/home/logo-vipkid.png	Vipkid

logo	alt
/images/home/logo-360.png	360

logo	alt
/images/home/logo-shopee.png	Shopee

logo	alt
/images/home/logo-tantan.png	探探

logo	alt
/images/home/logo-kuaishou.png	快手

logo	alt
/images/home/logo-sohu.png	搜狐

logo	alt
/images/home/logo-yidian.png	一点资讯

logo	alt
/images/home/logo-dingdong.png	叮咚买菜

logo	alt
/images/home/logo-youdao.png	有道

Quick Start →

/en/installing/compilation