[feature](file cache)Import file cache for remote file reader (#15622)
The main purpose of this pr is to import `fileCache` for lakehouse reading remote files. Use the local disk as the cache for reading remote file, so the next time this file is read, the data can be obtained directly from the local disk. In addition, this pr includes a few other minor changes Import File Cache: 1. The imported `fileCache` is called `block_file_cache`, which uses lru replacement policy. 2. Implement a new FileRereader `CachedRemoteFilereader`, so that the logic of `file cache` is hidden under `CachedRemoteFilereader`. Other changes: 1. Add a new interface `fs()` for `FileReader`. 2. `IOContext` adds some statistical information to count the situation of `FileCache` Co-authored-by: Lightman <31928846+Lchangliang@users.noreply.github.com>
This commit is contained in:
@ -372,7 +372,7 @@ private:
|
||||
|
||||
io::FileReaderSPtr create_readable_block(const std::string& path) {
|
||||
io::FileReaderSPtr reader;
|
||||
auto st = io::global_local_filesystem()->open_file(path, &reader);
|
||||
auto st = io::global_local_filesystem()->open_file(path, &reader, nullptr);
|
||||
return st.ok() ? std::move(reader) : nullptr;
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user