[enhancement](log4j) support high performance mode for log4j to escape potential bottleneck for doris read and write (#20759)
As we know, log4j2 some times may be bottleneck in doris fe when there are many logs to be output in sync mode while asynchronous logging has a better performance, and we find that capturing caller location has a similar impact across all logging libraries, and slows down asynchronous logging by about 30-100x. so, here we provide three log mode for log4j2 to meet the needs of different users. refer to https://logging.apache.org/log4j/2.x/performance.html
This commit is contained in:
@ -34,10 +34,10 @@ public class Config extends ConfigBase {
|
||||
* sys_log_dir:
|
||||
* This specifies FE log dir. FE will produces 2 log files:
|
||||
* fe.log: all logs of FE process.
|
||||
* fe.warn.log all WARNING and ERROR log of FE process.
|
||||
* fe.warn.log all WARN and ERROR log of FE process.
|
||||
*
|
||||
* sys_log_level:
|
||||
* INFO, WARNING, ERROR, FATAL
|
||||
* INFO, WARN, ERROR, FATAL
|
||||
*
|
||||
* sys_log_roll_num:
|
||||
* Maximal FE log files to be kept within an sys_log_roll_interval.
|
||||
@ -65,9 +65,18 @@ public class Config extends ConfigBase {
|
||||
"The path of the FE log file, used to store fe.log"})
|
||||
public static String sys_log_dir = System.getenv("DORIS_HOME") + "/log";
|
||||
|
||||
@ConfField(description = {"FE 日志的级别", "The level of FE log"}, options = {"INFO", "WARNING", "ERROR", "FATAL"})
|
||||
@ConfField(description = {"FE 日志的级别", "The level of FE log"}, options = {"INFO", "WARN", "ERROR", "FATAL"})
|
||||
public static String sys_log_level = "INFO";
|
||||
|
||||
@ConfField(description = {"FE 日志的输出模式,其中 NORMAL 为默认的输出模式,日志同步输出且包含位置信息,"
|
||||
+ "BRIEF 模式是日志同步输出但不包含位置信息,ASYNC 模式是日志异步输出且不包含位置信息,三种日志输出模式的性能依次递增",
|
||||
"The output mode of FE log, and NORMAL mode is the default output mode, which means the logs are "
|
||||
+ "synchronized and contain location information. BRIEF mode is synchronized and does not contain"
|
||||
+ " location information. ASYNC mode is asynchronous and does not contain location information."
|
||||
+ " The performance of the three log output modes increases in sequence"},
|
||||
options = {"NORMAL", "BRIEF", "ASYNC"})
|
||||
public static String sys_log_mode = "NORMAL";
|
||||
|
||||
@ConfField(description = {"FE 日志文件的最大数量。超过这个数量后,最老的日志文件会被删除",
|
||||
"The maximum number of FE log files. After exceeding this number, the oldest log file will be deleted"})
|
||||
public static int sys_log_roll_num = 10;
|
||||
|
||||
Reference in New Issue
Block a user