doris

Files

Mingyu Chen de66312a1a [Job Clean] Sort out the cleanup logic of historical jobs (#6553 )

There are many historical job records in Doris, such as load jobs, alter jobs, export jobs and so on.

These historical jobs are generally cleaned up periodically by the cleanup thread, to avoid taking too much memory.

This PR reorganized the cleanup logic of historical jobs and optimized the cleanup logic of some historical jobs
to reduce the memory usage of historical jobs.

The following FE configuration items are related to historical job cleaning:

1. label_keep_max_second

    Used to determine whether LoadJob, LoadJobV2, RoutineLoadJob or TransactionState are expired.

2. streaming_label_keep_max_second

    Used to determine whether InsertJob, DeleteJob  or TransactionState are expired.
    Different from label_keep_max_second, this config is used to clean up these frequently submitted jobs or load transactions.

3. history_job_keep_max_second

    Used to determine whether AlterJob, ExportJob are expired

2021-09-07 16:57:45 +08:00

conf

[Feature][LDAP] Add LDAP authentication login and LDAP group authorization support. (#6333 )

2021-07-30 09:24:50 +08:00

fe-common

[New Feature][Meta][Image] Add file header and footer for image (#6207 )

2021-07-27 13:36:53 +08:00

fe-core

[Job Clean] Sort out the cleanup logic of historical jobs (#6553 )

2021-09-07 16:57:45 +08:00

spark-dpp

fix(sparkload): bitmap deep copy in or operator (#6480 )

2021-09-02 12:15:02 +08:00

checkstyle-apache-header.txt

Add Checkstyle for doris-fe (#1353 )

2019-06-21 21:45:54 +08:00

checkstyle.xml

Add classes related to "tag". (#2343 )

2019-12-15 20:13:29 +08:00

pom.xml

[Doc] Modify README to add some statistical indicators (#6486 )

2021-08-25 09:36:26 +08:00

README

[CodeRefactor] Modify FE modules (#4146 )

2020-07-29 16:18:05 +08:00

README

# fe-common

This module is used to store some common classes of other modules.

# spark-dpp

This module is Spark DPP program, used for Spark Load function.
Depends: fe-common

# fe-core

This module is the main process module of FE.
Depends: fe-common, spark-dpp