Commit Graph

5 Commits

Author SHA1 Message Date
331fa50501 [feature](cold-data) move cold data to object storage without losing any feature(BE) (#10280)
This PR supports rowset level data upload on the BE side, so that there can be both cold data and hot data in a tablet,
and there is no necessary to prohibit loading new data to cooled tablets.

Each rowset is bound to a `FileSystem`, so that the storage layer can read and write rowsets without
perceiving the underlying filesystem.

The abstracted `RemoteFileSystem` can try local caching strategies with different granularity,
instead of caching segment files as before.

To avoid conflicts with the code in be/src/io, we temporarily put the file system related code in the be/src/io/fs directory.
In the future, `FileReader`s and `FileWriter`s should be unified.
2022-07-08 12:18:39 +08:00
6fedf5881b [CodeFormat] Clang-format cpp sources (#4965)
Clang-format all c++ source files.
2020-11-28 18:36:49 +08:00
6cbefd5621 [LRUCache] Expose LRU Cache status to metrics (#4688)
Expose LRU Cache status to metrics would be helpful to diagnose
problems like high usage, low hit rate.
2020-10-22 21:37:02 +08:00
2c8fdb6134 [BUG]Make segment V1 and V2 share same file cache (#3945)
This commit make segment V1 and V2 share on same file cache,
so that segment V2's file descriptors stored in cache can be cleaned up as V1 do.
2020-06-29 18:43:09 +08:00
a27e89065b Add file cache for v2 (#2782)
Add file descriptor cache for segment v2 to solve too many open file problems
2020-02-04 00:16:01 +08:00