doris

Files

hui lai 4b72a4c04e [improve](routine load) delay schedule EOF tasks to avoid too many small transactions (#39975 ) (#40509 )

pick (#39975)

We encountered a scenario where a large number of small transactions
were generated, resulting in an impact on query performance: Kafka's
data comes in batches of very small data every very short time, which
leads to tasks being frequently scheduled and ending very quickly,
resulting in a large number of small transactions.

To solve this problem, we delay the scheduling of tasks that perceive
EOF, which would not delay data consumption, for perceiving EOF
indicates that the consumption speed is greater than the production
speed.

2024-09-09 11:36:22 +08:00

.idea

…

be-java-extensions

[fix](multi-catalog)fix max compute null parts table read (#40046 ) (#40179 )

2024-08-30 22:56:08 +08:00

check/checkstyle

[fix](paimon) support paimon with hive2 (#32455 )

2024-03-26 15:31:07 +08:00

fe-common

[feature](metacache) add system table catalog_meta_cache_statistics #40155 (#40210 )

2024-09-02 23:23:35 +08:00

fe-core

[improve](routine load) delay schedule EOF tasks to avoid too many small transactions (#39975 ) (#40509 )

2024-09-09 11:36:22 +08:00

hive-udf

…

spark-dpp

[Improve](spark-load)update spark version for spark load to resolve cve problem (#30368 )

2024-03-21 14:07:23 +08:00

pom.xml

[branch-2.1](thirdparty) upgrade arrow to 17.0.0 (#39773 )

2024-08-22 18:35:32 +08:00

README

…

README

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.

# fe-common

This module is used to store some common classes of other modules.

# spark-dpp

This module is Spark DPP program, used for Spark Load function.
Depends: fe-common

# fe-core

This module is the main process module of FE.
Depends: fe-common, spark-dpp