database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

[refactor](file-system)(step-2) remove env, file_utils and filesystem_utils (#18009)

Browse Source 
Follow #17586.
This PR mainly changes:

Remove env/
Remove FileUtils/FilesystemUtils
Some methods are moved to LocalFileSystem
Remove olap/file_cache
Add s3 client cache for s3 file system
In my test, the time of open s3 file can be reduced significantly
Fix cold/hot separation bug for s3 fs.
This is the last PR of #17764.
After this, all IO operation should be in io/fs.

Except for tests in #17586, I also tested some case related to fs io:

clone
concurrency query on local/s3/hdfs
load error log create and clean
disk metrics
This commit is contained in:
Mingyu Chen
2023-03-29 09:00:52 +08:00
committed by GitHub
parent c3fe113894
commit 05db6e9b55
165 changed files with 1905 additions and 4835 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
be/test/runtime/user_function_cache_test.cpp
View File

@ -27,5 +27,4 @@
#include "http/http_channel.h"
#include "http/http_handler.h"
#include "http/http_request.h"
#include "util/file_utils.h"
#include "util/md5.h"