[improvement](s3) increase the connection num of s3 client (#22049)

The default maxConnection of s3 client is 25.
It should be increased to improve the query performance.

In my test, a tpch 300 benchmark with data stored on object storage, the total time
can reduce from 430s -> 330s
This commit is contained in:
Mingyu Chen
2023-07-26 22:52:40 +08:00
committed by GitHub
parent 560731f392
commit 7ed997cba8

View File

@ -145,7 +145,10 @@ std::shared_ptr<Aws::S3::S3Client> S3ClientFactory::create(const S3Conf& s3_conf
aws_config.region = s3_conf.region;
if (s3_conf.max_connections > 0) {
aws_config.maxConnections = s3_conf.max_connections;
} else {
aws_config.maxConnections = config::doris_remote_scanner_thread_pool_thread_num;
}
if (s3_conf.request_timeout_ms > 0) {
aws_config.requestTimeoutMs = s3_conf.request_timeout_ms;
}