database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

[fix][improvement](fs) add HdfsIO profile and modification time (#21638)

Browse Source 
Refactor the interface of create_file_reader

the file_size and mtime are merged into FileDescription, not in FileReaderOptions anymore.
Now the file handle cache can get correct file's modification time from FileDescription.
Add HdfsIO for hdfs file reader
pick from [Enhancement](multi-catalog) Add hdfs read statistics profile. #21442
This commit is contained in:
Mingyu Chen
2023-07-08 14:49:44 +08:00
committed by GitHub
parent 51b0bbb667
commit 2678afd2db
42 changed files with 269 additions and 327 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
be/src/olap/data_dir.cpp
View File

@ -39,7 +39,6 @@
#include "gutil/strings/substitute.h"
#include "io/fs/file_reader_writer_fwd.h"
#include "io/fs/file_writer.h"
#include "io/fs/fs_utils.h"
#include "io/fs/local_file_system.h"
#include "io/fs/path.h"
#include "io/fs/remote_file_system.h"
@ -149,7 +148,7 @@ Status DataDir::read_cluster_id(const std::string& cluster_id_path, int32_t* clu
if (exists) {
std::string content;
RETURN_IF_ERROR(
io::read_file_to_string(io::global_local_filesystem(), cluster_id_path, &content));
io::global_local_filesystem()->read_file_to_string(cluster_id_path, &content));
if (content.size() > 0) {
*cluster_id = std::stoi(content);
} else {