doris

Files

Mingyu Chen 51ccd44865 [Load Parallel][3/3] Support parallel delta writer (#5369 )

In the previous broker load, multiple OlapTableSinks would send data to the same LoadChannel,
and because of the lock granularity problem, LoadChannel could only process these requests serially,
which made it impossible to make full use of cluster resources.

This CL modifies the related locks so that LoadChannel can process these requests in parallel.

In the test, with a size of 20G, the load speed of 334 million rows of data in 3 nodes has been
increased from 9min to 5min, and after enabling 2 concurrency, it can be increased to 3min.

Also modify the profile of load job.

2021-02-07 22:42:18 +08:00

fe-common

[Enhancement] move common codes from fe-core to fe-common and remove log4j1 (#5317 ) (#5318 )

2021-02-04 13:41:03 +08:00

fe-core

[Load Parallel][3/3] Support parallel delta writer (#5369 )

2021-02-07 22:42:18 +08:00

spark-dpp

[Enhancement] move common codes from fe-core to fe-common and remove log4j1 (#5317 ) (#5318 )

2021-02-04 13:41:03 +08:00

checkstyle-apache-header.txt

Add Checkstyle for doris-fe (#1353 )

2019-06-21 21:45:54 +08:00

checkstyle.xml

Add classes related to "tag". (#2343 )

2019-12-15 20:13:29 +08:00

pom.xml

[Enhancement] move common codes from fe-core to fe-common and remove log4j1 (#5317 ) (#5318 )

2021-02-04 13:41:03 +08:00

README

[CodeRefactor] Modify FE modules (#4146 )

2020-07-29 16:18:05 +08:00

README

# fe-common

This module is used to store some common classes of other modules.

# spark-dpp

This module is Spark DPP program, used for Spark Load function.
Depends: fe-common

# fe-core

This module is the main process module of FE.
Depends: fe-common, spark-dpp