github-mirror/mergerfs
github-mirror/mergerfs
2
0
Fork 0
mirror of https://github.com/trapexit/mergerfs.git synced 2025-04-24 22:14:05 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

Merge branch 'master' into readdir_rewind_fix

Browse Source
This commit is contained in:
trapexit 2021-02-09 18:13:08 -05:00 committed by GitHub
commit cbea2ba91b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
330 changed files with 6970 additions and 3229 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
.github/FUNDING.yml vendored
View File

@ -2,6 +2,6 @@
github: [trapexit]
patreon: trapexit
open_collective: trapexit
custom: ['https://paypal.me/trapexit','https://buymeacoff.ee/trapexit','https://www.subscribestar.com/trapexit']
ko_fi: trapexit
custom: ['https://paypal.me/trapexit','https://www.subscribestar.com/trapexit']
open_collective: trapexit

2
LICENSE
View File

@ -1,7 +1,7 @@
/*
ISC License
Copyright (c) 2020, Antonio SJ Musumeci <trapexit@spawn.link>
Copyright (c) 2021, Antonio SJ Musumeci <trapexit@spawn.link>
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above

32
Makefile
View File

@ -58,13 +58,20 @@ LTO_FLAGS :=
endif
SRC = $(wildcard src/*.cpp)
OBJS = $(SRC:src/%.cpp=build/%.o)
DEPS = $(SRC:src/%.cpp=build/%.d)
OBJS = $(SRC:src/%.cpp=build/.src/%.o)
DEPS = $(SRC:src/%.cpp=build/.src/%.d)
TESTS = $(wildcard tests/*.cpp)
TESTS_OBJS = $(filter-out build/.src/mergerfs.o,$(OBJS))
TESTS_OBJS += $(TESTS:tests/%.cpp=build/.tests/%.o)
TESTS_DEPS = $(TESTS:tests/%.cpp=build/.tests/%.d)
TESTS_DEPS += $(DEPS)
MANPAGE = mergerfs.1
CXXFLAGS ?= ${OPT_FLAGS}
CXXFLAGS := \
${CXXFLAGS} \
-std=c++0x \
-std=c++11 \
$(STATIC_FLAGS) \
$(LTO_FLAGS) \
-Wall \
@ -77,6 +84,9 @@ FUSE_FLAGS = \
MFS_FLAGS = \
-DUSE_XATTR=$(USE_XATTR) \
-DUGID_USE_RWLOCK=$(UGID_USE_RWLOCK)
TESTS_FLAGS = \
-Isrc \
-DTESTS
LDFLAGS := \
${LDFLAGS} \
@ -110,11 +120,19 @@ help:
objects: version build/stamp
$(MAKE) $(OBJS)
tests-objects:
$(MAKE) $(TESTS_OBJS)
build/mergerfs: libfuse objects
$(CXX) $(CXXFLAGS) $(FUSE_FLAGS) $(MFS_FLAGS) $(CPPFLAGS) $(OBJS) -o $@ libfuse/build/libfuse.a $(LDFLAGS)
build/tests: build/mergerfs tests-objects
$(CXX) $(CXXFLAGS) $(TESTS_FLAGS) $(FUSE_FLAGS) $(MFS_FLAGS) $(CPPFLAGS) $(TESTS_OBJS) -o $@ libfuse/build/libfuse.a $(LDFLAGS)
mergerfs: build/mergerfs
tests: build/tests
changelog:
ifeq ($(GIT_REPO),1)
$(GIT2DEBCL) --name mergerfs > ChangeLog
@ -127,12 +145,16 @@ version:
tools/update-version
build/stamp:
$(MKDIR) -p build
$(MKDIR) -p build/.src build/.tests
$(TOUCH) $@
build/%.o: src/%.cpp
build/.src/%.o: src/%.cpp
$(CXX) $(CXXFLAGS) $(FUSE_FLAGS) $(MFS_FLAGS) $(CPPFLAGS) -c $< -o $@
build/.tests/%.o: tests/%.cpp
$(CXX) $(CXXFLAGS) $(TESTS_FLAGS) $(FUSE_FLAGS) $(MFS_FLAGS) $(CPPFLAGS) -c $< -o $@
.PHONY: clean
clean: rpm-clean
$(RM) -rf build

164
README.md
View File

@ -1,6 +1,6 @@
% mergerfs(1) mergerfs user manual
% Antonio SJ Musumeci <trapexit@spawn.link>
% 2020-08-30
% 2021-02-08
# NAME
@ -31,7 +31,7 @@ mergerfs -o&lt;options&gt; &lt;branches&gt; &lt;mountpoint&gt;
* Handles pool of read-only and read/write drives
* Can turn read-only files into symlinks to underlying file
* Hard link copy-on-write / CoW
* supports POSIX ACLs
* Support for POSIX ACLs
# HOW IT WORKS
@ -59,7 +59,7 @@ A + B = C
+-- file6
```
mergerfs does **not** support the copy-on-write (CoW) behavior found in **aufs** and **overlayfs**. You can **not** mount a read-only filesystem and write to it. However, mergerfs will ignore read-only drives when creating new files so you can mix read-write and read-only drives. It also does **not** split data across drives. It is not RAID0 / striping. It is simply a union.
mergerfs does **NOT** support the copy-on-write (CoW) or whiteout behaviors found in **aufs** and **overlayfs**. You can **not** mount a read-only filesystem and write to it. However, mergerfs will ignore read-only drives when creating new files so you can mix read-write and read-only drives. It also does **NOT** split data across drives. It is not RAID0 / striping. It is simply a union of other filesystems.
# TERMINOLOGY
@ -67,9 +67,9 @@ mergerfs does **not** support the copy-on-write (CoW) behavior found in **aufs**
* branch: A base path used in the pool.
* pool: The mergerfs mount. The union of the branches.
* relative path: The path in the pool relative to the branch and mount.
* function: A filesystem call (open, unlink, create, getattr, rmdir, etc.)
* category: A collection of functions based on basic behavior (action, create, search).
* policy: The algorithm used to select a file when performing a function.
* function: A filesystem call (open, unlink, create, getattr, etc.)
* category: A collection of functions (action, create, search).
* path preservation: Aspect of some policies which includes checking the path for which a file would be created.
@ -77,22 +77,25 @@ mergerfs does **not** support the copy-on-write (CoW) behavior found in **aufs**
If you don't already know that you have a special use case then just start with one of the following option sets.
#### You need `mmap` (used by rtorrent and many sqlite3 base software)
`allow_other,use_ino,cache.files=partial,dropcacheonclose=true,category.create=mfs`
#### You don't need `mmap`
`use_ino,cache.files=off,dropcacheonclose=true,allow_other,category.create=mfs`
`allow_other,use_ino,cache.files=off,dropcacheonclose=true,category.create=mfs`
#### You do need `mmap` (used by rtorrent and some other programs)
`use_ino,cache.files=partial,dropcacheonclose=true,allow_other,category.create=mfs`
See the mergerfs [wiki for real world deployments](https://github.com/trapexit/mergerfs/wiki/Real-World-Deployments) for comparisons / ideas.
# OPTIONS
These options are the same regardless you use them with the `mergerfs` commandline program, used in fstab, or in a config file.
### mount options
* **config**: Path to a config file. Same arguments as below in key=val format.
* **config**: Path to a config file. Same arguments as below in key=val / ini style format.
* **branches**: Colon delimited list of branches.
* **allow_other**: A libfuse option which allows users besides the one which ran mergerfs to see the filesystem. This is required for most use-cases.
* **minfreespace=SIZE**: The minimum space value used for creation policies. Can be overridden by branch specific option. Understands 'K', 'M', and 'G' to represent kilobyte, megabyte, and gigabyte respectively. (default: 4G)
@ -101,7 +104,7 @@ See the mergerfs [wiki for real world deployments](https://github.com/trapexit/m
* **inodecalc=passthrough|path-hash|devino-hash|hybrid-hash**: Selects the inode calculation algorithm. (default: hybrid-hash)
* **dropcacheonclose=BOOL**: When a file is requested to be closed call `posix_fadvise` on it first to instruct the kernel that we no longer need the data and it can drop its cache. Recommended when **cache.files=partial|full|auto-full** to limit double caching. (default: false)
* **symlinkify=BOOL**: When enabled and a file is not writable and its mtime or ctime is older than **symlinkify_timeout** files will be reported as symlinks to the original files. Please read more below before using. (default: false)
* **symlinkify_timeout=INT**: Time to wait, in seconds, to activate the **symlinkify** behavior. (default: 3600)
* **symlinkify_timeout=UINT**: Time to wait, in seconds, to activate the **symlinkify** behavior. (default: 3600)
* **nullrw=BOOL**: Turns reads and writes into no-ops. The request will succeed but do nothing. Useful for benchmarking mergerfs. (default: false)
* **ignorepponrename=BOOL**: Ignore path preserving on rename. Typically rename and link act differently depending on the policy of `create` (read below). Enabling this will cause rename and link to always use the non-path preserving behavior. This means files, when renamed or linked, will stay on the same drive. (default: false)
* **security_capability=BOOL**: If false return ENOATTR when xattr security.capability is queried. (default: true)
@ -112,16 +115,18 @@ See the mergerfs [wiki for real world deployments](https://github.com/trapexit/m
* **nfsopenhack=off|git|all**: A workaround for exporting mergerfs over NFS where there are issues with creating files for write while setting the mode to read-only. (default: off)
* **posix_acl=BOOL**: Enable POSIX ACL support (if supported by kernel and underlying filesystem). (default: false)
* **async_read=BOOL**: Perform reads asynchronously. If disabled or unavailable the kernel will ensure there is at most one pending read request per file handle and will attempt to order requests by offset. (default: true)
* **fuse_msg_size=INT**: Set the max number of pages per FUSE message. Only available on Linux >= 4.20 and ignored otherwise. (min: 1; max: 256; default: 256)
* **fuse_msg_size=UINT**: Set the max number of pages per FUSE message. Only available on Linux >= 4.20 and ignored otherwise. (min: 1; max: 256; default: 256)
* **threads=INT**: Number of threads to use in multithreaded mode. When set to zero it will attempt to discover and use the number of logical cores. If the lookup fails it will fall back to using 4. If the thread count is set negative it will look up the number of cores then divide by the absolute value. ie. threads=-2 on an 8 core machine will result in 8 / 2 = 4 threads. There will always be at least 1 thread. NOTE: higher number of threads increases parallelism but usually decreases throughput. (default: 0)
* **fsname=STR**: Sets the name of the filesystem as seen in **mount**, **df**, etc. Defaults to a list of the source paths concatenated together with the longest common prefix removed.
* **func.FUNC=POLICY**: Sets the specific FUSE function's policy. See below for the list of value types. Example: **func.getattr=newest**
* **category.CATEGORY=POLICY**: Sets policy of all FUSE functions in the provided category. See POLICIES section for defaults. Example: **category.create=mfs**
* **cache.open=INT**: 'open' policy cache timeout in seconds. (default: 0)
* **cache.statfs=INT**: 'statfs' cache timeout in seconds. (default: 0)
* **cache.attr=INT**: File attribute cache timeout in seconds. (default: 1)
* **cache.entry=INT**: File name lookup cache timeout in seconds. (default: 1)
* **cache.negative_entry=INT**: Negative file name lookup cache timeout in seconds. (default: 0)
* **category.action=POLICY**: Sets policy of all FUSE functions in the action category. (default: epall)
* **category.create=POLICY**: Sets policy of all FUSE functions in the create category. (default: epmfs)
* **category.search=POLICY**: Sets policy of all FUSE functions in the search category. (default: ff)
* **cache.open=UINT**: 'open' policy cache timeout in seconds. (default: 0)
* **cache.statfs=UINT**: 'statfs' cache timeout in seconds. (default: 0)
* **cache.attr=UINT**: File attribute cache timeout in seconds. (default: 1)
* **cache.entry=UINT**: File name lookup cache timeout in seconds. (default: 1)
* **cache.negative_entry=UINT**: Negative file name lookup cache timeout in seconds. (default: 0)
* **cache.files=libfuse|off|partial|full|auto-full**: File page caching mode (default: libfuse)
* **cache.writeback=BOOL**: Enable kernel writeback caching (default: false)
* **cache.symlinks=BOOL**: Cache symlinks (if supported by kernel) (default: false)
@ -139,17 +144,18 @@ See the mergerfs [wiki for real world deployments](https://github.com/trapexit/m
#### Value Types
* BOOL = 'true' | 'false'
* INT = [0,MAX_INT]
* INT = [MIN_INT,MAX_INT]
* UINT = [0,MAX_INT]
* SIZE = 'NNM'; NN = INT, M = 'K' | 'M' | 'G' | 'T'
* STR = string
* FUNC = FUSE function
* CATEGORY = FUSE function category
* FUNC = filesystem function
* CATEGORY = function category
* POLICY = mergerfs function policy
### branches
The 'branches' (formerly 'srcmounts') argument is a colon (':') delimited list of paths to be pooled together. It does not matter if the paths are on the same or different drives nor does it matter the filesystem (within reason). Used and available space will not be duplicated for paths on the same device and any features which aren't supported by the underlying filesystem (such as file attributes or extended attributes) will return the appropriate errors.
The 'branches' argument is a colon (':') delimited list of paths to be pooled together. It does not matter if the paths are on the same or different drives nor does it matter the filesystem (within reason). Used and available space will not be duplicated for paths on the same device and any features which aren't supported by the underlying filesystem (such as file attributes or extended attributes) will return the appropriate errors.
Branches currently have two options which can be set. A type which impacts whether or not the branch is included in a policy calculation and a individual minfreespace value. The values are set by prepending an `=` at the end of a branch designation and using commas as delimiters. Example: /mnt/drive=RW,1234
@ -270,25 +276,47 @@ This hack addresses the issue where the creation of a file with a read-only mode
Even though it's a more niche situation this hack breaks normal security and behavior and as such is `off` by default. If set to `git` it will only perform the hack when the path in question includes `/.git/`. `all` will result it it applying anytime a readonly file which is empty is opened for writing.
# FUNCTIONS / POLICIES / CATEGORIES
# FUNCTIONS, CATEGORIES and POLICIES
The POSIX filesystem API is made up of a number of functions. **creat**, **stat**, **chown**, etc. For ease of configuration in mergerfs most of the core functions are grouped into 3 categories: **action**, **create**, and **search**. These functions and categories can be assigned a policy which dictates which underlying branch/file/directory is chosen when performing that behavior. Any policy can be assigned to a function or category though some may not be very useful in practice. For instance: **rand** (random) may be useful for file creation (create) but could lead to very odd behavior if used for `chmod` if there were more than one copy of the file.
The POSIX filesystem API is made up of a number of functions. **creat**, **stat**, **chown**, etc. For ease of configuration in mergerfs most of the core functions are grouped into 3 categories: **action**, **create**, and **search**. These functions and categories can be assigned a policy which dictates which branch is chosen when performing that function.
Some functions, listed in the category `N/A` below, can not be assigned the normal policies. All functions which work on file handles use the handle which was acquired by `open` or `create`. `readdir` has no real need for a policy given the purpose is merely to return a list of entries in a directory. `statfs`'s behavior can be modified via other options. That said many times the current FUSE kernel driver will not always provide the file handle when a client calls `fgetattr`, `fchown`, `fchmod`, `futimens`, `ftruncate`, etc. This means it will call the regular, path based, versions.
Some functions, listed in the category `N/A` below, can not be assigned the normal policies. These functions work with file handles, rather than file paths, which were created by `open` or `create`. That said many times the current FUSE kernel driver will not always provide the file handle when a client calls `fgetattr`, `fchown`, `fchmod`, `futimens`, `ftruncate`, etc. This means it will call the regular, path based, versions. `readdir` has no real need for a policy given the purpose is merely to return a list of entries in a directory. `statfs`'s behavior can be modified via other options.
When using policies which are based on a branch's available space the base path provided is used. Not the full path to the file in question. Meaning that sub mounts won't be considered in the space calculations. The reason is that it doesn't really work for non-path preserving policies and can lead to non-obvious behaviors.
When using policies which are based on a branch's available space the base path provided is used. Not the full path to the file in question. Meaning that mounts in the branch won't be considered in the space calculations. The reason is that it doesn't really work for non-path preserving policies and can lead to non-obvious behaviors.
NOTE: While any policy can be assigned to a function or category though some may not be very useful in practice. For instance: **rand** (random) may be useful for file creation (create) but could lead to very odd behavior if used for `chmod` if there were more than one copy of the file.
#### Functions and their Category classifications
### Functions and their Category classifications
| Category | FUSE Functions |
|----------|-------------------------------------------------------------------------------------|
| action | chmod, chown, link, removexattr, rename, rmdir, setxattr, truncate, unlink, utimens |
| create | create, mkdir, mknod, symlink |
| search | access, getattr, getxattr, ioctl (directories), listxattr, open, readlink |
| search | access, getattr, getxattr, ioctl (directories), listxattr, open, readlink |
| N/A | fchmod, fchown, futimens, ftruncate, fallocate, fgetattr, fsync, ioctl (files), read, readdir, release, statfs, write, copy_file_range |
In cases where something may be searched (to confirm a directory exists across all source mounts) **getattr** will be used.
In cases where something may be searched for (such as a path to clone) **getattr** will usually be used.
### Policies
A policy is the algorithm used to choose a branch or branches for a function to work on. Think of them as ways to filter and sort branches.
Any function in the `create` category will clone the relative path if needed. Some other functions (`rename`,`link`,`ioctl`) have special requirements or behaviors which you can read more about below.
#### Filtering
Policies basically search branches and create a list of files / paths for functions to work on. The policy is responsible for filtering and sorting the branches. Filters include **minfreespace**, whether or not a branch is mounted read-only, and the branch tagging (RO,NC,RW). These filters are applied across all policies unless otherwise noted.
* No **search** function policies filter.
* All **action** function policies filter out branches which are mounted **read-only** or tagged as **RO (read-only)**.
* All **create** function policies filter out branches which are mounted **read-only**, tagged **RO (read-only)** or **NC (no create)**, or has available space less than `minfreespace`.
Policies may have their own additional filtering such as those that require existing paths to be present.
If all branches are filtered an error will be returned. Typically **EROFS** (read-only filesystem) or **ENOSPC** (no space left on device) depending on the most recent reason for filtering a branch. **ENOENT** will be returned if no elegible branch is found.
#### Path Preservation
@ -304,20 +332,9 @@ When using non-path preserving policies paths will be cloned to target drives as
With the `msp` or `most shared path` policies they are defined as `path preserving` for the purpose of controlling `link` and `rename`'s behaviors since `ignorepponrename` is available to disable that behavior. In mergerfs v3.0 the path preserving behavior of rename and link will likely be separated from the policy all together.
#### Filters
Policies basically search branches and create a list of files / paths for functions to work on. The policy is responsible for filtering and sorting. Filters include **minfreespace**, whether or not a branch is mounted read-only, and the branch tagging (RO,NC,RW). The policy defines the sorting but filtering is mostly uniform as described below.
* No **search** policies filter.
* All **action** policies will filter out branches which are mounted **read-only** or tagged as **RO (read-only)**.
* All **create** policies will filter out branches which are mounted **read-only**, tagged **RO (read-only)** or **NC (no create)**, or has available space less than `minfreespace`.
If all branches are filtered an error will be returned. Typically **EROFS** (read-only filesystem) or **ENOSPC** (no space left on device) depending on the most recent reason for filtering a branch.
#### Policy descriptions
Because of the nature of the behavior the policies act differently depending on the function it is used with (based on the category).
A policy's behavior differs, as mentioned above, based on the function it is used with. Sometimes it really might not make sense to even offer certain policies because they are literally the same as others but it makes things a bit more uniform. In mergerfs 3.0 this might change.
| Policy | Description |
@ -360,19 +377,6 @@ Because of the nature of the behavior the policies act differently depending on
When `ioctl` is used with an open file then it will use the file handle which was created at the original `open` call. However, when using `ioctl` with a directory mergerfs will use the `open` policy to find the directory to act on.
#### unlink
In FUSE there is an opaque "file handle" which is created by `open`, `create`, or `opendir`, passed to the kernel, and then is passed back to the FUSE userland application by the kernel. Unfortunately, the FUSE kernel driver does not always send the file handle when it theoretically could/should. This complicates certain behaviors / workflows particularly in the high level API. As a result mergerfs is currently doing a few hacky things.
libfuse2 and libfuse3, when using the high level API, will rename names to `.fuse_hiddenXXXXXX` if the file is open when unlinked or renamed over. It does this so the file is still available when a request referencing the now missing file is made. This file however keeps a `rmdir` from succeeding and can be picked up by software reading directories.
The change mergerfs has done is that if a file is open when an unlink or rename happens it will open the file and keep it open till closed by all those who opened it prior. When a request comes in referencing that file and it doesn't include a file handle it will instead use the file handle created at unlink/rename time.
This won't result in technically proper behavior but close enough for many usecases.
The plan is to rewrite mergerfs to use the low level API so these invasive libfuse changes are no longer necessary.
#### rename & link ####
**NOTE:** If you're receiving errors from software when files are moved / renamed / linked then you should consider changing the create policy to one which is **not** path preserving, enabling `ignorepponrename`, or contacting the author of the offending software and requesting that `EXDEV` (cross device / improper link) be properly handled.
@ -473,7 +477,7 @@ $ wget https://github.com/trapexit/mergerfs/releases/download/<ver>/mergerfs-<ve
$ cd mergerfs
$ sudo tools/install-build-pkgs
$ make deb
$ sudo dpkg -i ../mergerfs_version_arch.deb
$ sudo dpkg -i ../mergerfs_<version>_<arch>.deb
```
#### RHEL / CentOS /Fedora
@ -664,7 +668,6 @@ A B C
* mergerfs.dup: Ensure there are at least N copies of a file across the pool
* mergerfs.balance: Rebalance files across drives by moving them from the most filled to the least filled
* mergerfs.consolidate: move files within a single mergerfs directory to the drive with most free space
* mergerfs.mktrash: Creates FreeDesktop.org Trash specification compatible directories on a mergerfs mount
* https://github.com/trapexit/scorch
* scorch: A tool to help discover silent corruption of files and keep track of files
* https://github.com/trapexit/bbf
@ -764,6 +767,9 @@ With #2 one could use dm-cache as well but there is another solution which requi
Move files from cache to backing pool based only on the last time the file was accessed. Replace `-atime` with `-amin` if you want minutes rather than days. May want to use the `fadvise` / `--drop-cache` version of rsync or run rsync with the tool "nocache".
*NOTE:* The arguments to these scripts include the cache **drive**. Not the pool with the cache drive. You could have data loss if the source is the cache pool.
```
#!/bin/bash
@ -785,6 +791,8 @@ find "${CACHE}" -type f -atime +${N} -printf '%P\n' | \
Move the oldest file from the cache to the backing pool. Continue till below percentage threshold.
*NOTE:* The arguments to these scripts include the cache **drive**. Not the pool with the cache drive. You could have data loss if the source is the cache pool.
```
#!/bin/bash
@ -869,6 +877,8 @@ $ dd if=/mnt/mergerfs/1GB.file of=/dev/null bs=1M count=1024 iflag=nocache conv=
# TIPS / NOTES
* This document is very literal and thorough. Unless there is a bug things work as described. If a suspected feature isn't mentioned it doesn't exist.
* Ensure you're using the latest version. Few distros have the latest version.
* **use_ino** will only work when used with mergerfs 2.18.0 and above.
* Run mergerfs as `root` (with **allow_other**) unless you're merging paths which are owned by the same user otherwise strange permission issues may arise.
* https://github.com/trapexit/backup-and-recovery-howtos : A set of guides / howtos on creating a data storage system, backing it up, maintaining it, and recovering from failure.
@ -885,6 +895,7 @@ $ dd if=/mnt/mergerfs/1GB.file of=/dev/null bs=1M count=1024 iflag=nocache conv=
[https://github.com/trapexit/mergerfs/wiki/Kernel-Issues-&-Bugs](https://github.com/trapexit/mergerfs/wiki/Kernel-Issues-&-Bugs)
#### directory mtime is not being updated
Remember that the default policy for `getattr` is `ff`. The information for the first directory found will be returned. If it wasn't the directory which had been updated then it will appear outdated.
@ -1061,17 +1072,21 @@ The default create policy is `epmfs`. That is a path preserving algorithm. With
This catches a lot of new users off guard but changing the default would break the setup for many existing users. If you do not care about path preservation and wish your files to be spread across all your drives change to `mfs` or similar policy as described above. If you do want path preservation you'll need to perform the manual act of creating paths on the drives you want the data to land on before transferring your data. Setting `func.mkdir=epall` can simplify managing path preservation for `create`. Or use `func.mkdir=rand` if you're interested in just grouping together directory content by drive.
#### Do hard links work?
#### Do hardlinks work?
Yes. You need to use `use_ino` to support proper reporting of inodes but they work regardless. See also the option `inodecalc`.
What mergerfs does not do is fake hard links across branches. Read the section "rename & link" for how it works.
Remember that hardlinks will NOT work across devices. That includes between the original filesystem and a mergerfs pool, between two separate pools of the same underlying filesystems, or bind mounts of paths within the mergerfs pool. The latter is common when using Docker or Podman. Multiple volumes (bind mounts) to the same underlying filesystem are considered different devices. There is no way to link between them. You should mount in the highest directory in the mergerfs pool that includes all the paths you need if you want links to work.
#### Does mergerfs support CoW / copy-on-write?
#### Does mergerfs support CoW / copy-on-write / writes to read-only filesystems?
Not in the sense of a filesystem like BTRFS or ZFS nor in the overlayfs or aufs sense. It does offer a [cow-shell](http://manpages.ubuntu.com/manpages/bionic/man1/cow-shell.1.html) like hard link breaking (copy to temp file then rename over original) which can be useful when wanting to save space by hardlinking duplicate files but wish to treat each name as if it were a unique and separate file.
If you want to write to a read-only filesystem you should look at overlayfs. You can always include the overlayfs mount into a mergerfs pool.
#### Why can't I see my files / directories?
@ -1268,21 +1283,34 @@ This software is free to use and released under a very liberal license (ISC). Th
At the moment my preference would be GitHub Sponsors only because I am part of the matching program. That said please use whatever platform you prefer.
* PayPal: https://paypal.me/trapexit
* GitHub Sponsors: https://github.com/sponsors/trapexit
* PayPal: https://paypal.me/trapexit
* Patreon: https://www.patreon.com/trapexit
* SubscribeStar: https://www.subscribestar.com/trapexit
* Ko-Fi: https://ko-fi.com/trapexit
* Open Collective: https://opencollective.com/trapexit
* Bitcoin (BTC): 1DfoUd2m5WCxJAMvcFuvDpT4DR2gWX2PWb
* Bitcoin Cash (BCH): qrf257j0l09yxty4kur8dk2uma8p5vntdcpks72l8z
* Ethereum (ETH): 0xb486C0270fF75872Fc51d85879b9c15C380E66CA
* Litecoin (LTC): LW1rvHRPWtm2NUEMhJpP4DjHZY1FaJ1WYs
* Monero (XMR): 8AuU7PeK1fVhGP9yug8fdgKBssvUQoBVFKGhtT5DzWQt7fcTKC1SUx3Eb7xCAiVt3McWJp2Z9gX2wU7SPhh1GfWYBTCs6SS
* Basic Attention Token (BAT): 0xE651d4900B4C305284Da43E2e182e9abE149A87A
* Bitcoin (BTC): bc1qu537hqlnmn2wawx9n7nws0dlkz55h0cd93ny28
* Bitcoin Cash (BCH): bitcoincash:qqp0vh9v44us74gaggwjfv9y54zfjmmd7srlqxa3xt
* Bitcoin SV (BSV): 1FkFuxRtt3f8LbkpeUKRZq7gKJFzGSGgZV
* Bitcoin Gold (BTG): AaPuJgJeohPjkB3LxJM6NKGnaHoRJ8ieT3
* Litecoin (LTC): MJQzsHBdNnkyGqCFdcAdHYKugicBmfAXfQ
* Dogecoin (DOGE): DLJNLVe28vZ4SMQSxDJLBQBv57rGtUoWFh
* Ethereum (ETH): 0xB8d6d55c0319aacC327860d13f891427caEede7a
* Basic Attention Token (BAT): 0xB8d6d55c0319aacC327860d13f891427caEede7a
* Chainlink (LINK): 0xB8d6d55c0319aacC327860d13f891427caEede7a
* Reserve Rights (RSR): 0xB8d6d55c0319aacC327860d13f891427caEede7a
* Reef Finance (REEF): 0xB8d6d55c0319aacC327860d13f891427caEede7a
* Any ERC20 Token: 0xB8d6d55c0319aacC327860d13f891427caEede7a
* Ethereum Classic (ETC): 0x2B6054428e69a1201B6555f7a2aEc0Fba01EAD9F
* Dash (DASH): XvsFrohu8tbjA4E8p7xsc86E2ADxLHGXHL
* Monero (XMR): 45BBZMrJwPSaFwSoqLVNEggWR2BJJsXxz7bNz8FXnnFo3GyhVJFSCrCFSS7zYwDa9r1TmFmGMxQ2HTntuc11yZ9q1LeCE8f
* Filecoin (FIL): f1wpypkjcluufzo74yha7p67nbxepzizlroockgcy
* LBRY Credits (LBC): bFusyoZPkSuzM2Pr8mcthgvkymaosJZt5r
* Zcash (ZEC): t1ZwTgmbQF23DJrzqbAmw8kXWvU2xUkkhTt
* Zcoin (XZC): a8L5Vz35KdCQe7Y7urK2pcCGau7JsqZ5Gw
* Ripple (XRP): r9f6aoxaGD8aymxqH89Ke1PCUPkNiFdZZC
* Tezos (XTZ): tz1ZxerkbbALsuU9XGV9K9fFpuLWnKAGfc1C
* Zcash (ZEC): t1Zo1GGn2T3GrhKvgdtnTsTnWu6tCPaCaHG
* DigiByte (DGB): Sb8r1qTrryY9Sp4YkTE1eeKEGVzgArnE5N
* Namecoin (NMC): NDzb9FkoptGu5QbgetCkodJqo2zE1cTwyb
* Vertcoin (VTC): 3PYdhokAGXJwWrwHRoTywxG4iUDk6EHjKe
* Other crypto currencies: contact me for address

304
man/mergerfs.1
View File

@ -1,7 +1,7 @@
.\"t
.\" Automatically generated by Pandoc 1.19.2.4
.\"
.TH "mergerfs" "1" "2020\-08\-30" "mergerfs user manual" ""
.TH "mergerfs" "1" "2021\-02\-08" "mergerfs user manual" ""
.hy
.SH NAME
.PP
@ -42,7 +42,7 @@ Can turn read\-only files into symlinks to underlying file
.IP \[bu] 2
Hard link copy\-on\-write / CoW
.IP \[bu] 2
supports POSIX ACLs
Support for POSIX ACLs
.SH HOW IT WORKS
.PP
mergerfs logically merges multiple paths together.
@ -74,14 +74,14 @@ A\ \ \ \ \ \ \ \ \ +\ \ \ \ \ \ B\ \ \ \ \ \ \ \ =\ \ \ \ \ \ \ C
\f[]
.fi
.PP
mergerfs does \f[B]not\f[] support the copy\-on\-write (CoW) behavior
found in \f[B]aufs\f[] and \f[B]overlayfs\f[].
mergerfs does \f[B]NOT\f[] support the copy\-on\-write (CoW) or whiteout
behaviors found in \f[B]aufs\f[] and \f[B]overlayfs\f[].
You can \f[B]not\f[] mount a read\-only filesystem and write to it.
However, mergerfs will ignore read\-only drives when creating new files
so you can mix read\-write and read\-only drives.
It also does \f[B]not\f[] split data across drives.
It also does \f[B]NOT\f[] split data across drives.
It is not RAID0 / striping.
It is simply a union.
It is simply a union of other filesystems.
.SH TERMINOLOGY
.IP \[bu] 2
branch: A base path used in the pool.
@ -91,33 +91,39 @@ The union of the branches.
.IP \[bu] 2
relative path: The path in the pool relative to the branch and mount.
.IP \[bu] 2
function: A filesystem call (open, unlink, create, getattr, rmdir, etc.)
.IP \[bu] 2
category: A collection of functions based on basic behavior (action,
create, search).
.IP \[bu] 2
policy: The algorithm used to select a file when performing a function.
.IP \[bu] 2
function: A filesystem call (open, unlink, create, getattr, etc.)
.IP \[bu] 2
category: A collection of functions (action, create, search).
.IP \[bu] 2
path preservation: Aspect of some policies which includes checking the
path for which a file would be created.
.SH BASIC SETUP
.PP
If you don\[aq]t already know that you have a special use case then just
start with one of the following option sets.
.SS You need \f[C]mmap\f[] (used by rtorrent and many sqlite3 base
software)
.PP
\f[C]allow_other,use_ino,cache.files=partial,dropcacheonclose=true,category.create=mfs\f[]
.SS You don\[aq]t need \f[C]mmap\f[]
.PP
\f[C]use_ino,cache.files=off,dropcacheonclose=true,allow_other,category.create=mfs\f[]
.SS You do need \f[C]mmap\f[] (used by rtorrent and some other programs)
.PP
\f[C]use_ino,cache.files=partial,dropcacheonclose=true,allow_other,category.create=mfs\f[]
\f[C]allow_other,use_ino,cache.files=off,dropcacheonclose=true,category.create=mfs\f[]
.PP
See the mergerfs wiki for real world
deployments (https://github.com/trapexit/mergerfs/wiki/Real-World-Deployments)
for comparisons / ideas.
.SH OPTIONS
.PP
These options are the same regardless you use them with the
\f[C]mergerfs\f[] commandline program, used in fstab, or in a config
file.
.SS mount options
.IP \[bu] 2
\f[B]config\f[]: Path to a config file.
Same arguments as below in key=val format.
Same arguments as below in key=val / ini style format.
.IP \[bu] 2
\f[B]branches\f[]: Colon delimited list of branches.
.IP \[bu] 2
@ -163,7 +169,7 @@ be reported as symlinks to the original files.
Please read more below before using.
(default: false)
.IP \[bu] 2
\f[B]symlinkify_timeout=INT\f[]: Time to wait, in seconds, to activate
\f[B]symlinkify_timeout=UINT\f[]: Time to wait, in seconds, to activate
the \f[B]symlinkify\f[] behavior.
(default: 3600)
.IP \[bu] 2
@ -227,7 +233,7 @@ pending read request per file handle and will attempt to order requests
by offset.
(default: true)
.IP \[bu] 2
\f[B]fuse_msg_size=INT\f[]: Set the max number of pages per FUSE
\f[B]fuse_msg_size=UINT\f[]: Set the max number of pages per FUSE
message.
Only available on Linux >= 4.20 and ignored otherwise.
(min: 1; max: 256; default: 256)
@ -254,24 +260,32 @@ longest common prefix removed.
See below for the list of value types.
Example: \f[B]func.getattr=newest\f[]
.IP \[bu] 2
\f[B]category.CATEGORY=POLICY\f[]: Sets policy of all FUSE functions in
the provided category.
See POLICIES section for defaults.
Example: \f[B]category.create=mfs\f[]
\f[B]category.action=POLICY\f[]: Sets policy of all FUSE functions in
the action category.
(default: epall)
.IP \[bu] 2
\f[B]cache.open=INT\f[]: \[aq]open\[aq] policy cache timeout in seconds.
\f[B]category.create=POLICY\f[]: Sets policy of all FUSE functions in
the create category.
(default: epmfs)
.IP \[bu] 2
\f[B]category.search=POLICY\f[]: Sets policy of all FUSE functions in
the search category.
(default: ff)
.IP \[bu] 2
\f[B]cache.open=UINT\f[]: \[aq]open\[aq] policy cache timeout in
seconds.
(default: 0)
.IP \[bu] 2
\f[B]cache.statfs=INT\f[]: \[aq]statfs\[aq] cache timeout in seconds.
\f[B]cache.statfs=UINT\f[]: \[aq]statfs\[aq] cache timeout in seconds.
(default: 0)
.IP \[bu] 2
\f[B]cache.attr=INT\f[]: File attribute cache timeout in seconds.
\f[B]cache.attr=UINT\f[]: File attribute cache timeout in seconds.
(default: 1)
.IP \[bu] 2
\f[B]cache.entry=INT\f[]: File name lookup cache timeout in seconds.
\f[B]cache.entry=UINT\f[]: File name lookup cache timeout in seconds.
(default: 1)
.IP \[bu] 2
\f[B]cache.negative_entry=INT\f[]: Negative file name lookup cache
\f[B]cache.negative_entry=UINT\f[]: Negative file name lookup cache
timeout in seconds.
(default: 0)
.IP \[bu] 2
@ -315,22 +329,24 @@ setting.
.IP \[bu] 2
BOOL = \[aq]true\[aq] | \[aq]false\[aq]
.IP \[bu] 2
INT = [0,MAX_INT]
INT = [MIN_INT,MAX_INT]
.IP \[bu] 2
UINT = [0,MAX_INT]
.IP \[bu] 2
SIZE = \[aq]NNM\[aq]; NN = INT, M = \[aq]K\[aq] | \[aq]M\[aq] |
\[aq]G\[aq] | \[aq]T\[aq]
.IP \[bu] 2
STR = string
.IP \[bu] 2
FUNC = FUSE function
FUNC = filesystem function
.IP \[bu] 2
CATEGORY = FUSE function category
CATEGORY = function category
.IP \[bu] 2
POLICY = mergerfs function policy
.SS branches
.PP
The \[aq]branches\[aq] (formerly \[aq]srcmounts\[aq]) argument is a
colon (\[aq]:\[aq]) delimited list of paths to be pooled together.
The \[aq]branches\[aq] argument is a colon (\[aq]:\[aq]) delimited list
of paths to be pooled together.
It does not matter if the paths are on the same or different drives nor
does it matter the filesystem (within reason).
Used and available space will not be duplicated for paths on the same
@ -613,7 +629,7 @@ If set to \f[C]git\f[] it will only perform the hack when the path in
question includes \f[C]/.git/\f[].
\f[C]all\f[] will result it it applying anytime a readonly file which is
empty is opened for writing.
.SH FUNCTIONS / POLICIES / CATEGORIES
.SH FUNCTIONS, CATEGORIES and POLICIES
.PP
The POSIX filesystem API is made up of a number of functions.
\f[B]creat\f[], \f[B]stat\f[], \f[B]chown\f[], etc.
@ -621,34 +637,34 @@ For ease of configuration in mergerfs most of the core functions are
grouped into 3 categories: \f[B]action\f[], \f[B]create\f[], and
\f[B]search\f[].
These functions and categories can be assigned a policy which dictates
which underlying branch/file/directory is chosen when performing that
behavior.
Any policy can be assigned to a function or category though some may not
be very useful in practice.
For instance: \f[B]rand\f[] (random) may be useful for file creation
(create) but could lead to very odd behavior if used for \f[C]chmod\f[]
if there were more than one copy of the file.
which branch is chosen when performing that function.
.PP
Some functions, listed in the category \f[C]N/A\f[] below, can not be
assigned the normal policies.
All functions which work on file handles use the handle which was
acquired by \f[C]open\f[] or \f[C]create\f[].
\f[C]readdir\f[] has no real need for a policy given the purpose is
merely to return a list of entries in a directory.
\f[C]statfs\f[]\[aq]s behavior can be modified via other options.
These functions work with file handles, rather than file paths, which
were created by \f[C]open\f[] or \f[C]create\f[].
That said many times the current FUSE kernel driver will not always
provide the file handle when a client calls \f[C]fgetattr\f[],
\f[C]fchown\f[], \f[C]fchmod\f[], \f[C]futimens\f[], \f[C]ftruncate\f[],
etc.
This means it will call the regular, path based, versions.
\f[C]readdir\f[] has no real need for a policy given the purpose is
merely to return a list of entries in a directory.
\f[C]statfs\f[]\[aq]s behavior can be modified via other options.
.PP
When using policies which are based on a branch\[aq]s available space
the base path provided is used.
Not the full path to the file in question.
Meaning that sub mounts won\[aq]t be considered in the space
Meaning that mounts in the branch won\[aq]t be considered in the space
calculations.
The reason is that it doesn\[aq]t really work for non\-path preserving
policies and can lead to non\-obvious behaviors.
.PP
NOTE: While any policy can be assigned to a function or category though
some may not be very useful in practice.
For instance: \f[B]rand\f[] (random) may be useful for file creation
(create) but could lead to very odd behavior if used for \f[C]chmod\f[]
if there were more than one copy of the file.
.SS Functions and their Category classifications
.PP
.TS
@ -685,8 +701,44 @@ fchmod, fchown, futimens, ftruncate, fallocate, fgetattr, fsync, ioctl
T}
.TE
.PP
In cases where something may be searched (to confirm a directory exists
across all source mounts) \f[B]getattr\f[] will be used.
In cases where something may be searched for (such as a path to clone)
\f[B]getattr\f[] will usually be used.
.SS Policies
.PP
A policy is the algorithm used to choose a branch or branches for a
function to work on.
Think of them as ways to filter and sort branches.
.PP
Any function in the \f[C]create\f[] category will clone the relative
path if needed.
Some other functions (\f[C]rename\f[],\f[C]link\f[],\f[C]ioctl\f[]) have
special requirements or behaviors which you can read more about below.
.SS Filtering
.PP
Policies basically search branches and create a list of files / paths
for functions to work on.
The policy is responsible for filtering and sorting the branches.
Filters include \f[B]minfreespace\f[], whether or not a branch is
mounted read\-only, and the branch tagging (RO,NC,RW).
These filters are applied across all policies unless otherwise noted.
.IP \[bu] 2
No \f[B]search\f[] function policies filter.
.IP \[bu] 2
All \f[B]action\f[] function policies filter out branches which are
mounted \f[B]read\-only\f[] or tagged as \f[B]RO (read\-only)\f[].
.IP \[bu] 2
All \f[B]create\f[] function policies filter out branches which are
mounted \f[B]read\-only\f[], tagged \f[B]RO (read\-only)\f[] or \f[B]NC
(no create)\f[], or has available space less than \f[C]minfreespace\f[].
.PP
Policies may have their own additional filtering such as those that
require existing paths to be present.
.PP
If all branches are filtered an error will be returned.
Typically \f[B]EROFS\f[] (read\-only filesystem) or \f[B]ENOSPC\f[] (no
space left on device) depending on the most recent reason for filtering
a branch.
\f[B]ENOENT\f[] will be returned if no elegible branch is found.
.SS Path Preservation
.PP
Policies, as described below, are of two basic types.
@ -709,33 +761,14 @@ defined as \f[C]path\ preserving\f[] for the purpose of controlling
\f[C]ignorepponrename\f[] is available to disable that behavior.
In mergerfs v3.0 the path preserving behavior of rename and link will
likely be separated from the policy all together.
.SS Filters
.PP
Policies basically search branches and create a list of files / paths
for functions to work on.
The policy is responsible for filtering and sorting.
Filters include \f[B]minfreespace\f[], whether or not a branch is
mounted read\-only, and the branch tagging (RO,NC,RW).
The policy defines the sorting but filtering is mostly uniform as
described below.
.IP \[bu] 2
No \f[B]search\f[] policies filter.
.IP \[bu] 2
All \f[B]action\f[] policies will filter out branches which are mounted
\f[B]read\-only\f[] or tagged as \f[B]RO (read\-only)\f[].
.IP \[bu] 2
All \f[B]create\f[] policies will filter out branches which are mounted
\f[B]read\-only\f[], tagged \f[B]RO (read\-only)\f[] or \f[B]NC (no
create)\f[], or has available space less than \f[C]minfreespace\f[].
.PP
If all branches are filtered an error will be returned.
Typically \f[B]EROFS\f[] (read\-only filesystem) or \f[B]ENOSPC\f[] (no
space left on device) depending on the most recent reason for filtering
a branch.
.SS Policy descriptions
.PP
Because of the nature of the behavior the policies act differently
depending on the function it is used with (based on the category).
A policy\[aq]s behavior differs, as mentioned above, based on the
function it is used with.
Sometimes it really might not make sense to even offer certain policies
because they are literally the same as others but it makes things a bit
more uniform.
In mergerfs 3.0 this might change.
.PP
.TS
tab(@);
@ -930,38 +963,6 @@ When \f[C]ioctl\f[] is used with an open file then it will use the file
handle which was created at the original \f[C]open\f[] call.
However, when using \f[C]ioctl\f[] with a directory mergerfs will use
the \f[C]open\f[] policy to find the directory to act on.
.SS unlink
.PP
In FUSE there is an opaque "file handle" which is created by
\f[C]open\f[], \f[C]create\f[], or \f[C]opendir\f[], passed to the
kernel, and then is passed back to the FUSE userland application by the
kernel.
Unfortunately, the FUSE kernel driver does not always send the file
handle when it theoretically could/should.
This complicates certain behaviors / workflows particularly in the high
level API.
As a result mergerfs is currently doing a few hacky things.
.PP
libfuse2 and libfuse3, when using the high level API, will rename names
to \f[C]\&.fuse_hiddenXXXXXX\f[] if the file is open when unlinked or
renamed over.
It does this so the file is still available when a request referencing
the now missing file is made.
This file however keeps a \f[C]rmdir\f[] from succeeding and can be
picked up by software reading directories.
.PP
The change mergerfs has done is that if a file is open when an unlink or
rename happens it will open the file and keep it open till closed by all
those who opened it prior.
When a request comes in referencing that file and it doesn\[aq]t include
a file handle it will instead use the file handle created at
unlink/rename time.
.PP
This won\[aq]t result in technically proper behavior but close enough
for many usecases.
.PP
The plan is to rewrite mergerfs to use the low level API so these
invasive libfuse changes are no longer necessary.
.SS rename & link
.PP
\f[B]NOTE:\f[] If you\[aq]re receiving errors from software when files
@ -1152,7 +1153,7 @@ $\ wget\ https://github.com/trapexit/mergerfs/releases/download/<ver>/mergerfs\-
$\ cd\ mergerfs
$\ sudo\ tools/install\-build\-pkgs
$\ make\ deb
$\ sudo\ dpkg\ \-i\ ../mergerfs_version_arch.deb
$\ sudo\ dpkg\ \-i\ ../mergerfs_<version>_<arch>.deb
\f[]
.fi
.SS RHEL / CentOS /Fedora
@ -1449,9 +1450,6 @@ most filled to the least filled
mergerfs.consolidate: move files within a single mergerfs directory to
the drive with most free space
.IP \[bu] 2
mergerfs.mktrash: Creates FreeDesktop.org Trash specification compatible
directories on a mergerfs mount
.IP \[bu] 2
https://github.com/trapexit/scorch
.IP \[bu] 2
scorch: A tool to help discover silent corruption of files and keep
@ -1697,6 +1695,11 @@ Replace \f[C]\-atime\f[] with \f[C]\-amin\f[] if you want minutes rather
than days.
May want to use the \f[C]fadvise\f[] / \f[C]\-\-drop\-cache\f[] version
of rsync or run rsync with the tool "nocache".
.PP
\f[I]NOTE:\f[] The arguments to these scripts include the cache
\f[B]drive\f[].
Not the pool with the cache drive.
You could have data loss if the source is the cache pool.
.IP
.nf
\f[C]
@ -1719,6 +1722,11 @@ find\ "${CACHE}"\ \-type\ f\ \-atime\ +${N}\ \-printf\ \[aq]%P\\n\[aq]\ |\ \\
.PP
Move the oldest file from the cache to the backing pool.
Continue till below percentage threshold.
.PP
\f[I]NOTE:\f[] The arguments to these scripts include the cache
\f[B]drive\f[].
Not the pool with the cache drive.
You could have data loss if the source is the cache pool.
.IP
.nf
\f[C]
@ -1741,7 +1749,7 @@ do
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ head\ \-n\ 1\ |\ \\
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ cut\ \-d\[aq]\ \[aq]\ \-f2\-)
\ \ \ \ test\ \-n\ "${FILE}"
\ \ \ \ rsync\ \-axqHAXWES\ \-\-preallocate\ \-\-remove\-source\-files\ "${CACHE}/./${FILE}"\ "${BACKING}/"
\ \ \ \ rsync\ \-axqHAXWESR\ \-\-preallocate\ \-\-remove\-source\-files\ "${CACHE}/./${FILE}"\ "${BACKING}/"
done
\f[]
.fi
@ -1893,6 +1901,13 @@ $\ dd\ if=/mnt/mergerfs/1GB.file\ of=/dev/null\ bs=1M\ count=1024\ iflag=nocache
.fi
.SH TIPS / NOTES
.IP \[bu] 2
This document is very literal and thorough.
Unless there is a bug things work as described.
If a suspected feature isn\[aq]t mentioned it doesn\[aq]t exist.
.IP \[bu] 2
Ensure you\[aq]re using the latest version.
Few distros have the latest version.
.IP \[bu] 2
\f[B]use_ino\f[] will only work when used with mergerfs 2.18.0 and
above.
.IP \[bu] 2
@ -2305,7 +2320,7 @@ Setting \f[C]func.mkdir=epall\f[] can simplify managing path
preservation for \f[C]create\f[].
Or use \f[C]func.mkdir=rand\f[] if you\[aq]re interested in just
grouping together directory content by drive.
.SS Do hard links work?
.SS Do hardlinks work?
.PP
Yes.
You need to use \f[C]use_ino\f[] to support proper reporting of inodes
@ -2314,7 +2329,19 @@ See also the option \f[C]inodecalc\f[].
.PP
What mergerfs does not do is fake hard links across branches.
Read the section "rename & link" for how it works.
.SS Does mergerfs support CoW / copy\-on\-write?
.PP
Remember that hardlinks will NOT work across devices.
That includes between the original filesystem and a mergerfs pool,
between two separate pools of the same underlying filesystems, or bind
mounts of paths within the mergerfs pool.
The latter is common when using Docker or Podman.
Multiple volumes (bind mounts) to the same underlying filesystem are
considered different devices.
There is no way to link between them.
You should mount in the highest directory in the mergerfs pool that
includes all the paths you need if you want links to work.
.SS Does mergerfs support CoW / copy\-on\-write / writes to read\-only
filesystems?
.PP
Not in the sense of a filesystem like BTRFS or ZFS nor in the overlayfs
or aufs sense.
@ -2324,6 +2351,10 @@ like hard link breaking (copy to temp file then rename over original)
which can be useful when wanting to save space by hardlinking duplicate
files but wish to treat each name as if it were a unique and separate
file.
.PP
If you want to write to a read\-only filesystem you should look at
overlayfs.
You can always include the overlayfs mount into a mergerfs pool.
.SS Why can\[aq]t I see my files / directories?
.PP
It\[aq]s almost always a permissions issue.
@ -2691,36 +2722,63 @@ At the moment my preference would be GitHub Sponsors only because I am
part of the matching program.
That said please use whatever platform you prefer.
.IP \[bu] 2
PayPal: https://paypal.me/trapexit
.IP \[bu] 2
GitHub Sponsors: https://github.com/sponsors/trapexit
.IP \[bu] 2
Patreon: https://www.patreon.com/trapexit
PayPal: https://paypal.me/trapexit
.IP \[bu] 2
SubscribeStar: https://www.subscribestar.com/trapexit
Patreon: https://www.patreon.com/trapexit
.IP \[bu] 2
Ko\-Fi: https://ko\-fi.com/trapexit
.IP \[bu] 2
Open Collective: https://opencollective.com/trapexit
.IP \[bu] 2
Bitcoin (BTC): 1DfoUd2m5WCxJAMvcFuvDpT4DR2gWX2PWb
Bitcoin (BTC): bc1qu537hqlnmn2wawx9n7nws0dlkz55h0cd93ny28
.IP \[bu] 2
Bitcoin Cash (BCH): qrf257j0l09yxty4kur8dk2uma8p5vntdcpks72l8z
Bitcoin Cash (BCH):
bitcoincash:qqp0vh9v44us74gaggwjfv9y54zfjmmd7srlqxa3xt
.IP \[bu] 2
Ethereum (ETH): 0xb486C0270fF75872Fc51d85879b9c15C380E66CA
Bitcoin SV (BSV): 1FkFuxRtt3f8LbkpeUKRZq7gKJFzGSGgZV
.IP \[bu] 2
Litecoin (LTC): LW1rvHRPWtm2NUEMhJpP4DjHZY1FaJ1WYs
Bitcoin Gold (BTG): AaPuJgJeohPjkB3LxJM6NKGnaHoRJ8ieT3
.IP \[bu] 2
Litecoin (LTC): MJQzsHBdNnkyGqCFdcAdHYKugicBmfAXfQ
.IP \[bu] 2
Dogecoin (DOGE): DLJNLVe28vZ4SMQSxDJLBQBv57rGtUoWFh
.IP \[bu] 2
Ethereum (ETH): 0xB8d6d55c0319aacC327860d13f891427caEede7a
.IP \[bu] 2
Basic Attention Token (BAT): 0xB8d6d55c0319aacC327860d13f891427caEede7a
.IP \[bu] 2
Chainlink (LINK): 0xB8d6d55c0319aacC327860d13f891427caEede7a
.IP \[bu] 2
Reserve Rights (RSR): 0xB8d6d55c0319aacC327860d13f891427caEede7a
.IP \[bu] 2
Reef Finance (REEF): 0xB8d6d55c0319aacC327860d13f891427caEede7a
.IP \[bu] 2
Any ERC20 Token: 0xB8d6d55c0319aacC327860d13f891427caEede7a
.IP \[bu] 2
Ethereum Classic (ETC): 0x2B6054428e69a1201B6555f7a2aEc0Fba01EAD9F
.IP \[bu] 2
Dash (DASH): XvsFrohu8tbjA4E8p7xsc86E2ADxLHGXHL
.IP \[bu] 2
Monero (XMR):
8AuU7PeK1fVhGP9yug8fdgKBssvUQoBVFKGhtT5DzWQt7fcTKC1SUx3Eb7xCAiVt3McWJp2Z9gX2wU7SPhh1GfWYBTCs6SS
45BBZMrJwPSaFwSoqLVNEggWR2BJJsXxz7bNz8FXnnFo3GyhVJFSCrCFSS7zYwDa9r1TmFmGMxQ2HTntuc11yZ9q1LeCE8f
.IP \[bu] 2
Basic Attention Token (BAT): 0xE651d4900B4C305284Da43E2e182e9abE149A87A
Filecoin (FIL): f1wpypkjcluufzo74yha7p67nbxepzizlroockgcy
.IP \[bu] 2
LBRY Credits (LBC): bFusyoZPkSuzM2Pr8mcthgvkymaosJZt5r
.IP \[bu] 2
Zcash (ZEC): t1ZwTgmbQF23DJrzqbAmw8kXWvU2xUkkhTt
Ripple (XRP): r9f6aoxaGD8aymxqH89Ke1PCUPkNiFdZZC
.IP \[bu] 2
Zcoin (XZC): a8L5Vz35KdCQe7Y7urK2pcCGau7JsqZ5Gw
Tezos (XTZ): tz1ZxerkbbALsuU9XGV9K9fFpuLWnKAGfc1C
.IP \[bu] 2
Zcash (ZEC): t1Zo1GGn2T3GrhKvgdtnTsTnWu6tCPaCaHG
.IP \[bu] 2
DigiByte (DGB): Sb8r1qTrryY9Sp4YkTE1eeKEGVzgArnE5N
.IP \[bu] 2
Namecoin (NMC): NDzb9FkoptGu5QbgetCkodJqo2zE1cTwyb
.IP \[bu] 2
Vertcoin (VTC): 3PYdhokAGXJwWrwHRoTywxG4iUDk6EHjKe
.IP \[bu] 2
Other crypto currencies: contact me for address
.SH LINKS

366
src/branch.cpp
View File

@ -18,21 +18,8 @@
#include "branch.hpp"
#include "ef.hpp"
#include "from_string.hpp"
#include "fs_glob.hpp"
#include "fs_realpathize.hpp"
#include "nonstd/optional.hpp"
#include "errno.hpp"
#include "num.hpp"
#include "str.hpp"
#include <string>
#include <errno.h>
#include <fnmatch.h>
using std::string;
using std::vector;
using nonstd::optional;
Branch::Branch(const uint64_t &default_minfreespace_)
@ -108,354 +95,3 @@ Branch::ro_or_nc(void) const
return ((mode == Branch::Mode::RO) ||
(mode == Branch::Mode::NC));
}
namespace l
{
static
void
split(const std::string &s_,
std::string *instr_,
std::string *values_)
{
uint64_t offset;
offset = s_.find_first_of('/');
*instr_ = s_.substr(0,offset);
if(offset != std::string::npos)
*values_ = s_.substr(offset);
}
}
Branches::Branches(const uint64_t &default_minfreespace_)
: default_minfreespace(default_minfreespace_)
{
pthread_rwlock_init(&lock,NULL);
}
namespace l
{
static
int
parse_mode(const string &str_,
Branch::Mode *mode_)
{
if(str_ == "RW")
*mode_ = Branch::Mode::RW;
ef(str_ == "RO")
*mode_ = Branch::Mode::RO;
ef(str_ == "NC")
*mode_ = Branch::Mode::NC;
else
return -EINVAL;
return 0;
}
static
int
parse_minfreespace(const string &str_,
optional<uint64_t> *minfreespace_)
{
int rv;
uint64_t uint64;
rv = str::from(str_,&uint64);
if(rv < 0)
return rv;
*minfreespace_ = uint64;
return 0;
}
static
int
parse_branch(const string &str_,
string *glob_,
Branch::Mode *mode_,
optional<uint64_t> *minfreespace_)
{
int rv;
string options;
vector<string> v;
str::rsplit1(str_,'=',&v);
switch(v.size())
{
case 1:
*glob_ = v[0];
*mode_ = Branch::Mode::RW;
break;
case 2:
*glob_ = v[0];
options = v[1];
v.clear();
str::split(options,',',&v);
switch(v.size())
{
case 2:
rv = l::parse_minfreespace(v[1],minfreespace_);
if(rv < 0)
return rv;
case 1:
rv = l::parse_mode(v[0],mode_);
if(rv < 0)
return rv;
break;
case 0:
return -EINVAL;
}
break;
default:
return -EINVAL;
}
return 0;
}
static
int
parse(const string &str_,
const uint64_t &default_minfreespace_,
BranchVec *branches_)
{
int rv;
string glob;
vector<string> globbed;
optional<uint64_t> minfreespace;
Branch branch(default_minfreespace_);
rv = l::parse_branch(str_,&glob,&branch.mode,&minfreespace);
if(rv < 0)
return rv;
if(minfreespace.has_value())
branch.set_minfreespace(minfreespace.value());
fs::glob(glob,&globbed);
fs::realpathize(&globbed);
for(size_t i = 0; i < globbed.size(); i++)
{
branch.path = globbed[i];
branches_->push_back(branch);
}
return 0;
}
static
int
set(const std::string &str_,
Branches *branches_)
{
int rv;
vector<string> paths;
BranchVec tmp_branchvec;
str::split(str_,':',&paths);
for(size_t i = 0; i < paths.size(); i++)
{
rv = l::parse(paths[i],branches_->default_minfreespace,&tmp_branchvec);
if(rv < 0)
return rv;
}
branches_->vec.clear();
branches_->vec.insert(branches_->vec.end(),
tmp_branchvec.begin(),
tmp_branchvec.end());
return 0;
}
static
int
add_begin(const std::string &str_,
Branches *branches_)
{
int rv;
vector<string> paths;
BranchVec tmp_branchvec;
str::split(str_,':',&paths);
for(size_t i = 0; i < paths.size(); i++)
{
rv = l::parse(paths[i],branches_->default_minfreespace,&tmp_branchvec);
if(rv < 0)
return rv;
}
branches_->vec.insert(branches_->vec.begin(),
tmp_branchvec.begin(),
tmp_branchvec.end());
return 0;
}
static
int
add_end(const std::string &str_,
Branches *branches_)
{
int rv;
vector<string> paths;
BranchVec tmp_branchvec;
str::split(str_,':',&paths);
for(size_t i = 0; i < paths.size(); i++)
{
rv = l::parse(paths[i],branches_->default_minfreespace,&tmp_branchvec);
if(rv < 0)
return rv;
}
branches_->vec.insert(branches_->vec.end(),
tmp_branchvec.begin(),
tmp_branchvec.end());
return 0;
}
static
int
erase_begin(BranchVec *branches_)
{
branches_->erase(branches_->begin());
return 0;
}
static
int
erase_end(BranchVec *branches_)
{
branches_->pop_back();
return 0;
}
static
int
erase_fnmatch(const std::string &str_,
Branches *branches_)
{
vector<string> patterns;
str::split(str_,':',&patterns);
for(BranchVec::iterator i = branches_->vec.begin();
i != branches_->vec.end();)
{
int match = FNM_NOMATCH;
for(vector<string>::const_iterator pi = patterns.begin();
pi != patterns.end() && match != 0;
++pi)
{
match = ::fnmatch(pi->c_str(),i->path.c_str(),0);
}
i = ((match == 0) ? branches_->vec.erase(i) : (i+1));
}
return 0;
}
}
int
Branches::from_string(const std::string &s_)
{
rwlock::WriteGuard guard(lock);
std::string instr;
std::string values;
l::split(s_,&instr,&values);
if(instr == "+")
return l::add_end(values,this);
if(instr == "+<")
return l::add_begin(values,this);
if(instr == "+>")
return l::add_end(values,this);
if(instr == "-")
return l::erase_fnmatch(values,this);
if(instr == "-<")
return l::erase_begin(&vec);
if(instr == "->")
return l::erase_end(&vec);
if(instr == "=")
return l::set(values,this);
if(instr.empty())
return l::set(values,this);
return -EINVAL;
}
string
Branches::to_string(void) const
{
rwlock::ReadGuard guard(lock);
string tmp;
for(size_t i = 0; i < vec.size(); i++)
{
const Branch &branch = vec[i];
tmp += branch.to_string();
tmp += ':';
}
if(*tmp.rbegin() == ':')
tmp.erase(tmp.size() - 1);
return tmp;
}
void
Branches::to_paths(vector<string> &vec_) const
{
rwlock::ReadGuard guard(lock);
for(size_t i = 0; i < vec.size(); i++)
{
const Branch &branch = vec[i];
vec_.push_back(branch.path);
}
}
SrcMounts::SrcMounts(Branches &b_)
: _branches(b_)
{
}
int
SrcMounts::from_string(const std::string &s_)
{
return _branches.from_string(s_);
}
std::string
SrcMounts::to_string(void) const
{
rwlock::ReadGuard guard(_branches.lock);
std::string rv;
for(uint64_t i = 0; i < _branches.vec.size(); i++)
{
rv += _branches.vec[i].path;
rv += ':';
}
if(*rv.rbegin() == ':')
rv.erase(rv.size() - 1);
return rv;
}

75
src/branch.hpp
View File

@ -18,81 +18,50 @@
#pragma once
#include "rwlock.hpp"
#include "tofrom_string.hpp"
#include "nonstd/optional.hpp"
#include "strvec.hpp"
#include "tofrom_string.hpp"
#include <cstdint>
#include <string>
#include <vector>
#include <stdint.h>
#include <pthread.h>
class Branch : public ToFromString
class Branch final : public ToFromString
{
public:
Branch(const uint64_t &default_minfreespace);
typedef std::vector<Branch> Vector;
public:
int from_string(const std::string &str);
std::string to_string(void) const;
Branch(const uint64_t &default_minfreespace_);
public:
enum class Mode
{
INVALID,
RO,
RW,
NC
INVALID,
RO,
RW,
NC
};
public:
Mode mode;
std::string path;
uint64_t minfreespace() const;
public:
void set_minfreespace(const uint64_t minfreespace);
public:
bool ro(void) const;
bool nc(void) const;
bool ro_or_nc(void) const;
public:
int from_string(const std::string &str) final;
std::string to_string(void) const final;
public:
uint64_t minfreespace() const;
void set_minfreespace(const uint64_t);
public:
Mode mode;
std::string path;
private:
nonstd::optional<uint64_t> _minfreespace;
const uint64_t *_default_minfreespace;
};
typedef std::vector<Branch> BranchVec;
class Branches : public ToFromString
{
public:
Branches(const uint64_t &default_minfreespace_);
public:
int from_string(const std::string &str);
std::string to_string(void) const;
public:
void to_paths(std::vector<std::string> &vec) const;
public:
mutable pthread_rwlock_t lock;
BranchVec vec;
const uint64_t &default_minfreespace;
};
class SrcMounts : public ToFromString
{
public:
SrcMounts(Branches &b_);
public:
int from_string(const std::string &str);
std::string to_string(void) const;
private:
Branches &_branches;
};

429
src/branches.cpp Normal file
View File

@ -0,0 +1,429 @@
/*
ISC License
Copyright (c) 2021, Antonio SJ Musumeci <trapexit@spawn.link>
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "branches.hpp"
#include "ef.hpp"
#include "errno.hpp"
#include "from_string.hpp"
#include "fs_glob.hpp"
#include "fs_realpathize.hpp"
#include "nonstd/optional.hpp"
#include "num.hpp"
#include "str.hpp"
#include <string>
#include <fnmatch.h>
using std::string;
using std::vector;
using nonstd::optional;
Branches::Impl::Impl(const uint64_t &default_minfreespace_)
: _default_minfreespace(default_minfreespace_)
{
}
Branches::Impl&
Branches::Impl::operator=(Branches::Impl &rval_)
{
auto this_base = dynamic_cast<Branch::Vector*>(this);
auto rval_base = dynamic_cast<Branch::Vector*>(&rval_);
*this_base = *rval_base;
return *this;
}
Branches::Impl&
Branches::Impl::operator=(Branches::Impl &&rval_)
{
auto this_base = dynamic_cast<Branch::Vector*>(this);
auto rval_base = dynamic_cast<Branch::Vector*>(&rval_);
*this_base = std::move(*rval_base);
return *this;
}
const
uint64_t&
Branches::Impl::minfreespace(void) const
{
return _default_minfreespace;
}
namespace l
{
static
void
split(const std::string &s_,
std::string *instr_,
std::string *values_)
{
uint64_t offset;
offset = s_.find_first_of('/');
*instr_ = s_.substr(0,offset);
if(offset != std::string::npos)
*values_ = s_.substr(offset);
}
static
int
parse_mode(const string &str_,
Branch::Mode *mode_)
{
if(str_ == "RW")
*mode_ = Branch::Mode::RW;
ef(str_ == "RO")
*mode_ = Branch::Mode::RO;
ef(str_ == "NC")
*mode_ = Branch::Mode::NC;
else
return -EINVAL;
return 0;
}
static
int
parse_minfreespace(const string &str_,
optional<uint64_t> *minfreespace_)
{
int rv;
uint64_t uint64;
rv = str::from(str_,&uint64);
if(rv < 0)
return rv;
*minfreespace_ = uint64;
return 0;
}
static
int
parse_branch(const string &str_,
string *glob_,
Branch::Mode *mode_,
optional<uint64_t> *minfreespace_)
{
int rv;
string options;
vector<string> v;
str::rsplit1(str_,'=',&v);
switch(v.size())
{
case 1:
*glob_ = v[0];
*mode_ = Branch::Mode::RW;
break;
case 2:
*glob_ = v[0];
options = v[1];
v.clear();
str::split(options,',',&v);
switch(v.size())
{
case 2:
rv = l::parse_minfreespace(v[1],minfreespace_);
if(rv < 0)
return rv;
case 1:
rv = l::parse_mode(v[0],mode_);
if(rv < 0)
return rv;
break;
case 0:
return -EINVAL;
}
break;
default:
return -EINVAL;
}
return 0;
}
static
int
parse(const string &str_,
Branches::Impl *branches_)
{
int rv;
string glob;
StrVec paths;
optional<uint64_t> minfreespace;
Branch branch(branches_->minfreespace());
rv = l::parse_branch(str_,&glob,&branch.mode,&minfreespace);
if(rv < 0)
return rv;
if(minfreespace.has_value())
branch.set_minfreespace(minfreespace.value());
fs::glob(glob,&paths);
fs::realpathize(&paths);
for(auto &path : paths)
{
branch.path = path;
branches_->push_back(branch);
}
return 0;
}
static
int
set(const std::string &str_,
Branches::Impl *branches_)
{
int rv;
StrVec paths;
Branches::Impl tmp_branches(branches_->minfreespace());
str::split(str_,':',&paths);
for(auto &path : paths)
{
rv = l::parse(path,&tmp_branches);
if(rv < 0)
return rv;
}
*branches_ = std::move(tmp_branches);
return 0;
}
static
int
add_begin(const std::string &str_,
Branches::Impl *branches_)
{
int rv;
vector<string> paths;
Branches::Impl tmp_branches(branches_->minfreespace());
str::split(str_,':',&paths);
for(auto &path : paths)
{
rv = l::parse(path,&tmp_branches);
if(rv < 0)
return rv;
}
branches_->insert(branches_->begin(),
tmp_branches.begin(),
tmp_branches.end());
return 0;
}
static
int
add_end(const std::string &str_,
Branches::Impl *branches_)
{
int rv;
StrVec paths;
Branches::Impl tmp_branches(branches_->minfreespace());
str::split(str_,':',&paths);
for(auto &path : paths)
{
rv = l::parse(path,&tmp_branches);
if(rv < 0)
return rv;
}
branches_->insert(branches_->end(),
tmp_branches.begin(),
tmp_branches.end());
return 0;
}
static
int
erase_begin(Branches::Impl *branches_)
{
branches_->erase(branches_->begin());
return 0;
}
static
int
erase_end(Branches::Impl *branches_)
{
branches_->pop_back();
return 0;
}
static
int
erase_fnmatch(const std::string &str_,
Branches::Impl *branches_)
{
StrVec patterns;
str::split(str_,':',&patterns);
for(auto i = branches_->begin(); i != branches_->end();)
{
int match = FNM_NOMATCH;
for(auto pi = patterns.cbegin(); pi != patterns.cend() && match != 0; ++pi)
{
match = ::fnmatch(pi->c_str(),i->path.c_str(),0);
}
i = ((match == 0) ? branches_->erase(i) : (i+1));
}
return 0;
}
}
int
Branches::Impl::from_string(const std::string &s_)
{
std::string instr;
std::string values;
l::split(s_,&instr,&values);
if(instr == "+")
return l::add_end(values,this);
if(instr == "+<")
return l::add_begin(values,this);
if(instr == "+>")
return l::add_end(values,this);
if(instr == "-")
return l::erase_fnmatch(values,this);
if(instr == "-<")
return l::erase_begin(this);
if(instr == "->")
return l::erase_end(this);
if(instr == "=")
return l::set(values,this);
if(instr.empty())
return l::set(values,this);
return -EINVAL;
}
std::string
Branches::Impl::to_string(void) const
{
string tmp;
if(empty())
return tmp;
for(auto &branch : *this)
{
tmp += branch.to_string();
tmp += ':';
}
tmp.pop_back();
return tmp;
}
void
Branches::Impl::to_paths(StrVec &paths_) const
{
for(auto &branch : *this)
{
paths_.push_back(branch.path);
}
}
int
Branches::from_string(const std::string &str_)
{
int rv;
Branches::Ptr impl;
Branches::Ptr new_impl;
{
std::lock_guard<std::mutex> lock_guard(_mutex);
impl = _impl;
}
new_impl = std::make_shared<Branches::Impl>(impl->minfreespace());
*new_impl = *impl;
rv = new_impl->from_string(str_);
if(rv < 0)
return rv;
{
std::lock_guard<std::mutex> lock_guard(_mutex);
_impl = new_impl;
}
return 0;
}
string
Branches::to_string(void) const
{
std::lock_guard<std::mutex> lock_guard(_mutex);
return _impl->to_string();
}
SrcMounts::SrcMounts(Branches &b_)
: _branches(b_)
{
}
int
SrcMounts::from_string(const std::string &s_)
{
return _branches.from_string(s_);
}
std::string
SrcMounts::to_string(void) const
{
std::string rv;
Branches::CPtr branches = _branches;
if(branches->empty())
return rv;
for(const auto &branch : *branches)
{
rv += branch.path;
rv += ':';
}
rv.pop_back();
return rv;
}

94
src/branches.hpp Normal file
View File

@ -0,0 +1,94 @@
/*
ISC License
Copyright (c) 2021, Antonio SJ Musumeci <trapexit@spawn.link>
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#pragma once
#include "branch.hpp"
#include "nonstd/optional.hpp"
#include "strvec.hpp"
#include "tofrom_string.hpp"
#include <cstdint>
#include <memory>
#include <mutex>
#include <string>
#include <vector>
class Branches final : public ToFromString
{
public:
class Impl final : public ToFromString, public Branch::Vector
{
public:
typedef std::shared_ptr<Impl> Ptr;
typedef std::shared_ptr<const Impl> CPtr;
public:
Impl(const uint64_t &default_minfreespace_);
public:
int from_string(const std::string &str) final;
std::string to_string(void) const final;
public:
const uint64_t& minfreespace(void) const;
void to_paths(StrVec &strvec) const;
public:
Impl& operator=(Impl &impl_);
Impl& operator=(Impl &&impl_);
private:
const uint64_t &_default_minfreespace;
};
public:
typedef Branches::Impl::Ptr Ptr;
typedef Branches::Impl::CPtr CPtr;
public:
Branches(const uint64_t &default_minfreespace_)
: _impl(std::make_shared<Impl>(default_minfreespace_))
{}
public:
int from_string(const std::string &str) final;
std::string to_string(void) const final;
public:
operator CPtr() const { std::lock_guard<std::mutex> lg(_mutex); return _impl; }
CPtr operator->() const { std::lock_guard<std::mutex> lg(_mutex); return _impl; }
private:
mutable std::mutex _mutex;
Ptr _impl;
};
class SrcMounts : public ToFromString
{
public:
SrcMounts(Branches &b_);
public:
int from_string(const std::string &str) final;
std::string to_string(void) const final;
private:
Branches &_branches;
};

49
src/category.cpp Normal file
View File

@ -0,0 +1,49 @@
/*
ISC License
Copyright (c) 2020, Antonio SJ Musumeci <trapexit@spawn.link>
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "category.hpp"
#include "errno.hpp"
#include "str.hpp"
#include <string>
int
Category::Base::from_string(const std::string &s_)
{
int rv;
for(auto func : funcs)
{
rv = func->from_string(s_);
if(rv < 0)
return rv;
}
return 0;
}
std::string
Category::Base::to_string(void) const
{
std::set<std::string> rv;
for(auto func : funcs)
rv.insert(func->to_string());
return str::join(rv,',');
}

93
src/category.hpp
View File

@ -1,5 +1,5 @@
/*
Copyright (c) 2016, Antonio SJ Musumeci <trapexit@spawn.link>
Copyright (c) 2020, Antonio SJ Musumeci <trapexit@spawn.link>
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
@ -16,9 +16,92 @@
#pragma once
enum class Category
#include "tofrom_string.hpp"
#include "funcs.hpp"
#include "func.hpp"
#include <string>
namespace Category
{
class Base : public ToFromString
{
ACTION,
CREATE,
SEARCH
public:
int from_string(const std::string &s) final;
std::string to_string() const final;
protected:
std::vector<ToFromString*> funcs;
};
class Action final : public Base
{
private:
Action();
public:
Action(Funcs &funcs_)
{
funcs.push_back(&funcs_.chmod);
funcs.push_back(&funcs_.chown);
funcs.push_back(&funcs_.link);
funcs.push_back(&funcs_.removexattr);
funcs.push_back(&funcs_.rename);
funcs.push_back(&funcs_.rmdir);
funcs.push_back(&funcs_.setxattr);
funcs.push_back(&funcs_.truncate);
funcs.push_back(&funcs_.unlink);
funcs.push_back(&funcs_.utimens);
}
};
class Create final : public Base
{
private:
Create();
public:
Create(Funcs &funcs_)
{
funcs.push_back(&funcs_.create);
funcs.push_back(&funcs_.mkdir);
funcs.push_back(&funcs_.mknod);
funcs.push_back(&funcs_.symlink);
}
};
class Search final : public Base
{
private:
Search();
public:
Search(Funcs &funcs_)
{
funcs.push_back(&funcs_.access);
funcs.push_back(&funcs_.getattr);
funcs.push_back(&funcs_.getxattr);
funcs.push_back(&funcs_.listxattr);
funcs.push_back(&funcs_.open);
funcs.push_back(&funcs_.readlink);
}
};
}
class Categories final
{
private:
Categories();
public:
Categories(Funcs &funcs_)
: action(funcs_),
create(funcs_),
search(funcs_)
{}
public:
Category::Action action;
Category::Create create;
Category::Search search;
};

228
src/config.cpp
View File

@ -20,14 +20,18 @@
#include "from_string.hpp"
#include "num.hpp"
#include "rwlock.hpp"
#include "str.hpp"
#include "to_string.hpp"
#include "version.hpp"
#include <algorithm>
#include <string>
#include <cstdint>
#include <fstream>
#include <iostream>
#include <string>
#include <stdint.h>
#include <pthread.h>
#include <string.h>
#include <sys/stat.h>
#include <unistd.h>
@ -37,6 +41,11 @@ using std::string;
#define IFERT(S) if(S == s_) return true
const std::string CONTROLFILE = "/.mergerfs";
Config Config::_singleton;
namespace l
{
static
@ -60,49 +69,44 @@ namespace l
}
Config::Config()
:
open_cache(),
controlfile("/.mergerfs"),
async_read(true),
auto_cache(false),
branches(minfreespace),
cache_attr(1),
cache_entry(1),
cache_files(CacheFiles::ENUM::LIBFUSE),
cache_negative_entry(0),
cache_readdir(false),
cache_statfs(0),
cache_symlinks(false),
category(func),
direct_io(false),
dropcacheonclose(false),
fsname(),
func(),
fuse_msg_size(FUSE_MAX_MAX_PAGES),
ignorepponrename(false),
inodecalc("hybrid-hash"),
link_cow(false),
minfreespace(MINFREESPACE_DEFAULT),
mount(),
moveonenospc(false),
nfsopenhack(NFSOpenHack::ENUM::OFF),
nullrw(false),
pid(::getpid()),
posix_acl(false),
readdir(ReadDir::ENUM::POSIX),
readdirplus(false),
security_capability(true),
srcmounts(branches),
statfs(StatFS::ENUM::BASE),
statfs_ignore(StatFSIgnore::ENUM::NONE),
symlinkify(false),
symlinkify_timeout(3600),
threads(0),
version(MERGERFS_VERSION),
writeback_cache(false),
xattr(XAttr::ENUM::PASSTHROUGH)
: async_read(true),
auto_cache(false),
branches(minfreespace),
cache_attr(1),
cache_entry(1),
cache_files(CacheFiles::ENUM::LIBFUSE),
cache_negative_entry(0),
cache_readdir(false),
cache_statfs(0),
cache_symlinks(false),
category(func),
direct_io(false),
dropcacheonclose(false),
fsname(),
func(),
fuse_msg_size(FUSE_MAX_MAX_PAGES),
ignorepponrename(false),
inodecalc("hybrid-hash"),
link_cow(false),
minfreespace(MINFREESPACE_DEFAULT),
mount(),
moveonenospc(false),
nfsopenhack(NFSOpenHack::ENUM::OFF),
nullrw(false),
pid(::getpid()),
posix_acl(false),
readdir(ReadDir::ENUM::POSIX),
readdirplus(false),
security_capability(true),
srcmounts(branches),
statfs(StatFS::ENUM::BASE),
statfs_ignore(StatFSIgnore::ENUM::NONE),
symlinkify(false),
symlinkify_timeout(3600),
threads(0),
version(MERGERFS_VERSION),
writeback_cache(false),
xattr(XAttr::ENUM::PASSTHROUGH)
{
_map["async_read"] = &async_read;
_map["auto_cache"] = &auto_cache;
@ -166,17 +170,22 @@ Config::Config()
_map["xattr"] = &xattr;
}
const
Config&
Config::ro(void)
Config::operator=(const Config &cfg_)
{
return *((Config*)fuse_get_context()->private_data);
}
int rv;
std::string val;
Config&
Config::rw(void)
{
return *((Config*)fuse_get_context()->private_data);
for(auto &kv : _map)
{
rv = cfg_.get(kv.first,&val);
if(rv)
continue;
kv.second->from_string(val);
}
return *this;
}
bool
@ -254,11 +263,80 @@ Config::set(const std::string &key_,
const std::string &value_)
{
if(l::readonly(key_))
return -EINVAL;
return -EROFS;
return set_raw(key_,value_);
}
int
Config::set(const std::string &kv_)
{
std::string key;
std::string val;
str::splitkv(kv_,'=',&key,&val);
key = str::trim(key);
val = str::trim(val);
return set(key,val);
}
int
Config::from_stream(std::istream &istrm_,
ErrVec *errs_)
{
int rv;
std::string line;
std::string key;
std::string val;
Config newcfg;
newcfg = *this;
while(std::getline(istrm_,line,'\n'))
{
line = str::trim(line);
if(!line.empty() && (line[0] == '#'))
continue;
str::splitkv(line,'=',&key,&val);
key = str::trim(key);
val = str::trim(val);
rv = newcfg.set(key,val);
if(rv < 0)
errs_->push_back({rv,key});
}
if(!errs_->empty())
return -EINVAL;
*this = newcfg;
return 0;
}
int
Config::from_file(const std::string &filepath_,
ErrVec *errs_)
{
int rv;
std::ifstream ifstrm;
ifstrm.open(filepath_);
if(!ifstrm.good())
{
errs_->push_back({-errno,filepath_});
return -errno;
}
rv = from_stream(ifstrm,errs_);
ifstrm.close();
return rv;
}
std::ostream&
operator<<(std::ostream &os_,
const Config &c_)
@ -268,7 +346,47 @@ operator<<(std::ostream &os_,
for(i = c_._map.begin(), ei = c_._map.end(); i != ei; ++i)
{
os_ << i->first << '=' << i->second << '\n';
os_ << i->first << '=' << i->second->to_string() << std::endl;
}
return os_;
}
static
std::string
err2str(const int err_)
{
switch(err_)
{
case 0:
return std::string();
case -EINVAL:
return "invalid value";
case -ENOATTR:
return "unknown option";
case -EROFS:
return "read-only option";
default:
return strerror(-err_);
}
return std::string();
}
std::ostream&
operator<<(std::ostream &os_,
const Config::ErrVec &ev_)
{
std::string errstr;
for(auto &err : ev_)
{
os_ << "* ERROR: ";
errstr = err2str(err.err);
if(!errstr.empty())
os_ << errstr << " - ";
os_ << err.str << std::endl;
}
return os_;

100
src/config.hpp
View File

@ -16,7 +16,8 @@
#pragma once
#include "branch.hpp"
#include "branches.hpp"
#include "category.hpp"
#include "config_cachefiles.hpp"
#include "config_inodecalc.hpp"
#include "config_moveonenospc.hpp"
@ -27,18 +28,20 @@
#include "config_xattr.hpp"
#include "enum.hpp"
#include "errno.hpp"
#include "func_category.hpp"
#include "funcs.hpp"
#include "policy.hpp"
#include "policy_cache.hpp"
#include "rwlock.hpp"
#include "tofrom_wrapper.hpp"
#include <fuse.h>
#include "fuse.h"
#include <cstdint>
#include <map>
#include <memory>
#include <mutex>
#include <string>
#include <vector>
#include <stdint.h>
#include <sys/stat.h>
typedef ToFromWrapper<bool> ConfigBOOL;
@ -47,16 +50,50 @@ typedef ToFromWrapper<int> ConfigINT;
typedef ToFromWrapper<std::string> ConfigSTR;
typedef std::map<std::string,ToFromString*> Str2TFStrMap;
extern const std::string CONTROLFILE;
class Config
{
public:
struct Err
{
int err;
std::string str;
};
typedef std::vector<Err> ErrVec;
public:
class Read
{
public:
Read();
public:
inline const Config* operator->() const;
private:
const Config &_cfg;
};
public:
class Write
{
public:
Write();
public:
Config* operator->();
private:
Config &_cfg;
};
public:
Config();
public:
mutable PolicyCache open_cache;
public:
const std::string controlfile;
Config& operator=(const Config&);
public:
ConfigBOOL async_read;
@ -69,7 +106,7 @@ public:
ConfigBOOL cache_readdir;
ConfigUINT64 cache_statfs;
ConfigBOOL cache_symlinks;
FuncCategories category;
Categories category;
ConfigBOOL direct_io;
ConfigBOOL dropcacheonclose;
ConfigSTR fsname;
@ -112,11 +149,50 @@ public:
int get(const std::string &key, std::string *val) const;
int set_raw(const std::string &key, const std::string &val);
int set(const std::string &key, const std::string &val);
int set(const std::string &kv);
public:
static const Config &ro(void);
static Config &rw(void);
int from_stream(std::istream &istrm, ErrVec *errs);
int from_file(const std::string &filepath, ErrVec *errs);
private:
Str2TFStrMap _map;
private:
static Config _singleton;
public:
friend class Read;
friend class Write;
};
std::ostream& operator<<(std::ostream &s,const Config::ErrVec &ev);
inline
Config::Read::Read()
: _cfg(Config::_singleton)
{
}
inline
const
Config*
Config::Read::operator->() const
{
return &_cfg;
}
inline
Config::Write::Write()
: _cfg(Config::_singleton)
{
}
inline
Config*
Config::Write::operator->()
{
return &_cfg;
}

1
src/config_cachefiles.hpp
View File

@ -20,6 +20,7 @@
#include "enum.hpp"
enum class CacheFilesEnum
{
LIBFUSE,

1
src/config_inodecalc.cpp
View File

@ -19,6 +19,7 @@
#include "config_inodecalc.hpp"
#include "fs_inode.hpp"
InodeCalc::InodeCalc(const std::string &s_)
{
fs::inode::set_algo(s_);

5
src/config_inodecalc.hpp
View File

@ -20,12 +20,13 @@
#include "tofrom_string.hpp"
class InodeCalc : public ToFromString
{
public:
InodeCalc(const std::string &);
public:
std::string to_string(void) const;
int from_string(const std::string &);
std::string to_string(void) const final;
int from_string(const std::string &) final;
};

9
src/config_moveonenospc.cpp
View File

@ -21,12 +21,13 @@
#include "errno.hpp"
#include "from_string.hpp"
int
MoveOnENOSPC::from_string(const std::string &s_)
{
int rv;
std::string s;
const Policy *tmp;
Policy::CreateImpl *tmp;
rv = str::from(s_,&enabled);
if((rv == 0) && (enabled == true))
@ -36,8 +37,8 @@ MoveOnENOSPC::from_string(const std::string &s_)
else
return 0;
tmp = &Policy::find(s);
if(tmp == Policy::invalid)
tmp = Policies::Create::find(s);
if(tmp == NULL)
return -EINVAL;
policy = tmp;
@ -50,6 +51,6 @@ std::string
MoveOnENOSPC::to_string(void) const
{
if(enabled)
return policy->to_string();
return policy.name();
return "false";
}

14
src/config_moveonenospc.hpp
View File

@ -19,26 +19,26 @@
#pragma once
#include "policy.hpp"
#include "policies.hpp"
#include "tofrom_string.hpp"
#include <string>
class MoveOnENOSPC : public ToFromString
{
public:
MoveOnENOSPC(const bool enabled_)
: enabled(enabled_)
: enabled(enabled_),
policy(&Policies::Create::mfs)
{
policy = (enabled ?
&Policy::mfs :
&Policy::invalid);
}
public:
int from_string(const std::string &s);
std::string to_string() const;
int from_string(const std::string &s) final;
std::string to_string() const final;
public:
bool enabled;
const Policy *policy;
Policy::Create policy;
};

1
src/config_nfsopenhack.cpp
View File

@ -20,6 +20,7 @@
#include "ef.hpp"
#include "errno.hpp"
template<>
int
NFSOpenHack::from_string(const std::string &s_)

1
src/config_nfsopenhack.hpp
View File

@ -20,6 +20,7 @@
#include "enum.hpp"
enum class NFSOpenHackEnum
{
OFF,

1
src/config_readdir.cpp
View File

@ -20,6 +20,7 @@
#include "ef.hpp"
#include "errno.hpp"
template<>
int
ReadDir::from_string(const std::string &s_)

1
src/config_readdir.hpp
View File

@ -20,6 +20,7 @@
#include "enum.hpp"
enum class ReadDirEnum
{
POSIX,

1
src/config_statfs.cpp
View File

@ -20,6 +20,7 @@
#include "ef.hpp"
#include "errno.hpp"
template<>
std::string
StatFS::to_string() const

1
src/config_statfs.hpp
View File

@ -20,6 +20,7 @@
#include "enum.hpp"
enum class StatFSEnum
{
BASE,

1
src/config_statfsignore.cpp
View File

@ -20,6 +20,7 @@
#include "ef.hpp"
#include "errno.hpp"
template<>
std::string
StatFSIgnore::to_string() const

1
src/config_xattr.cpp
View File

@ -20,6 +20,7 @@
#include "ef.hpp"
#include "errno.hpp"
template<>
std::string
XAttr::to_string() const

1
src/config_xattr.hpp
View File

@ -21,6 +21,7 @@
#include "enum.hpp"
#include "errno.hpp"
enum class XAttrEnum
{
PASSTHROUGH = 0,

1
src/dirinfo.hpp
View File

@ -20,6 +20,7 @@
#include <string>
class DirInfo : public FH
{
public:

1
src/endian.hpp
View File

@ -18,6 +18,7 @@
#pragma once
namespace endian
{
static

1
src/enum.hpp
View File

@ -22,6 +22,7 @@
#include <string>
template<typename E>
class Enum : public ToFromString
{

1
src/fh.hpp
View File

@ -18,6 +18,7 @@
#include <string>
class FH
{
public:

1
src/fileinfo.hpp
View File

@ -20,6 +20,7 @@
#include <string>
class FileInfo : public FH
{
public:

4
src/fixed_mem_pool.hpp
View File

@ -18,8 +18,10 @@
#pragma once
#include <cstdint>
#include <stdlib.h>
#include <stdint.h>
typedef struct fixed_mem_pool_t fixed_mem_pool_t;
struct fixed_mem_pool_t

17
src/from_string.cpp
View File

@ -17,13 +17,14 @@
*/
#include "ef.hpp"
#include "errno.hpp"
#include <cstdint>
#include <string>
#include <errno.h>
#include <stdint.h>
#include <stdlib.h>
namespace str
{
int
@ -50,7 +51,17 @@ namespace str
from(const std::string &value_,
int *int_)
{
*int_ = ::strtol(value_.c_str(),NULL,10);
int tmp;
char *endptr;
errno = 0;
tmp = ::strtol(value_.c_str(),&endptr,10);
if(errno != 0)
return -EINVAL;
if(endptr == value_.c_str())
return -EINVAL;
*int_ = tmp;
return 0;
}

2
src/from_string.hpp
View File

@ -18,9 +18,9 @@
#pragma once
#include <cstdint>
#include <string>
#include <stdint.h>
namespace str
{

1
src/fs_acl.cpp
View File

@ -23,6 +23,7 @@
const char POSIX_ACL_DEFAULT_XATTR[] = "system.posix_acl_default";
namespace fs
{
namespace acl

1
src/fs_acl.hpp
View File

@ -20,6 +20,7 @@
#include <string>
namespace fs
{
namespace acl

1
src/fs_attr.hpp
View File

@ -18,6 +18,7 @@
#include <string>
namespace fs
{
namespace attr

1
src/fs_attr_linux.icpp
View File

@ -26,6 +26,7 @@
using std::string;
namespace fs
{
namespace attr

1
src/fs_attr_unsupported.icpp
View File

@ -18,6 +18,7 @@
#include <string>
namespace fs
{
namespace attr

1
src/fs_clonefile.cpp
View File

@ -22,6 +22,7 @@
#include "fs_fchmod.hpp"
#include "fs_futimens.hpp"
namespace l
{
static

1
src/fs_clonefile.hpp
View File

@ -16,6 +16,7 @@
#pragma once
namespace fs
{
int

5
src/fs_clonepath.cpp
View File

@ -14,8 +14,6 @@
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <string>
#include "errno.h"
#include "fs_attr.hpp"
#include "fs_clonepath.hpp"
@ -27,8 +25,11 @@
#include "fs_xattr.hpp"
#include "ugid.hpp"
#include <string>
using std::string;
namespace l
{
static

1
src/fs_clonepath.hpp
View File

@ -18,6 +18,7 @@
#include <string>
namespace fs
{
int clonepath(const std::string &from,

1
src/fs_close.hpp
View File

@ -20,6 +20,7 @@
#include <unistd.h>
namespace fs
{
static

1
src/fs_closedir.hpp
View File

@ -21,6 +21,7 @@
#include <dirent.h>
#include <sys/types.h>
namespace fs
{
static

4
src/fs_copy_file_range.hpp
View File

@ -18,10 +18,12 @@
#pragma once
#include <cstdint>
#include <stddef.h>
#include <stdint.h>
#include <sys/types.h>
namespace fs
{
int64_t

4
src/fs_copy_file_range_linux.icpp
View File

@ -22,13 +22,15 @@
#include "errno.hpp"
#include <cstdint>
#include <fcntl.h>
#include <stdint.h>
#include <stdio.h>
#include <sys/syscall.h>
#include <sys/types.h>
#include <unistd.h>
namespace l
{
static

4
src/fs_copy_file_range_unsupported.icpp
View File

@ -18,10 +18,12 @@
#include "errno.h"
#include <cstdint>
#include <stddef.h>
#include <stdint.h>
#include <sys/types.h>
namespace fs
{
ssize_t

1
src/fs_copydata.cpp
View File

@ -24,6 +24,7 @@
#include <stddef.h>
namespace fs
{
int

1
src/fs_copydata.hpp
View File

@ -20,6 +20,7 @@
#include <stddef.h>
namespace fs
{
int

3
src/fs_copydata_copy_file_range.cpp
View File

@ -18,7 +18,8 @@
#include "fs_copy_file_range.hpp"
#include "fs_fstat.hpp"
#include <stdint.h>
#include <cstdint>
namespace l
{

3
src/fs_copydata_copy_file_range.hpp
View File

@ -16,7 +16,8 @@
#pragma once
#include <stdint.h>
#include <cstdint>
namespace fs
{

1
src/fs_copydata_readwrite.cpp
View File

@ -24,6 +24,7 @@
using std::vector;
namespace l
{
static

1
src/fs_copydata_readwrite.hpp
View File

@ -16,6 +16,7 @@
#pragma once
namespace fs
{
int

1
src/fs_cow.cpp
View File

@ -34,6 +34,7 @@
using std::string;
namespace l
{
static

1
src/fs_cow.hpp
View File

@ -21,6 +21,7 @@
#include <sys/stat.h>
#include <sys/types.h>
namespace fs
{
namespace cow

1
src/fs_devid.hpp
View File

@ -20,6 +20,7 @@
#include "fs_fstat.hpp"
namespace fs
{
static

1
src/fs_dirfd.hpp
View File

@ -21,6 +21,7 @@
#include <dirent.h>
#include <sys/types.h>
namespace fs
{
static

1
src/fs_dup.hpp
View File

@ -20,6 +20,7 @@
#include <unistd.h>
namespace fs
{
static

1
src/fs_eaccess.hpp
View File

@ -20,6 +20,7 @@
#include "fs_faccessat.hpp"
namespace fs
{
static

1
src/fs_exists.hpp
View File

@ -23,6 +23,7 @@
#include <string>
namespace fs
{
static

1
src/fs_faccessat.hpp
View File

@ -23,6 +23,7 @@
#include <fcntl.h>
#include <unistd.h>
namespace fs
{
static

1
src/fs_fadvise.hpp
View File

@ -18,6 +18,7 @@
#include <sys/types.h>
namespace fs
{
int

1
src/fs_fadvise_posix.icpp
View File

@ -16,6 +16,7 @@
#include <fcntl.h>
namespace fs
{
static

1
src/fs_fadvise_unsupported.icpp
View File

@ -16,6 +16,7 @@
#include "errno.hpp"
namespace fs
{
static

1
src/fs_fallocate.hpp
View File

@ -18,6 +18,7 @@
#include <fcntl.h>
namespace fs
{
int

1
src/fs_fallocate_linux.icpp
View File

@ -16,6 +16,7 @@
#include <fcntl.h>
namespace fs
{
int

3
src/fs_fallocate_osx.icpp
View File

@ -14,10 +14,11 @@
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "errno.hpp"
#include <fcntl.h>
#include <unistd.h>
#include "errno.hpp"
namespace l
{

1
src/fs_fallocate_posix.icpp
View File

@ -18,6 +18,7 @@
#include <fcntl.h>
namespace fs
{
int

1
src/fs_fallocate_unsupported.icpp
View File

@ -16,6 +16,7 @@
#include "errno.hpp"
namespace fs
{
int

1
src/fs_fchmod.hpp
View File

@ -24,6 +24,7 @@
#define MODE_BITS (S_ISUID|S_ISGID|S_ISVTX|S_IRWXU|S_IRWXG|S_IRWXO)
namespace fs
{
static

1
src/fs_fchmodat.hpp
View File

@ -23,6 +23,7 @@
#include <fcntl.h>
#include <sys/stat.h>
namespace fs
{
static

1
src/fs_fchown.hpp
View File

@ -25,6 +25,7 @@
#include <sys/types.h>
#include <unistd.h>
namespace fs
{
static

1
src/fs_fdatasync.hpp
View File

@ -26,6 +26,7 @@
#include <unistd.h>
namespace fs
{
static

1
src/fs_fgetxattr.hpp
View File

@ -25,6 +25,7 @@
#include <sys/types.h>
namespace fs
{
static

1
src/fs_ficlone.hpp
View File

@ -18,6 +18,7 @@
#pragma once
namespace fs
{
int

1
src/fs_ficlone_linux.icpp
View File

@ -21,6 +21,7 @@
#include <linux/fs.h>
namespace fs
{
int

1
src/fs_ficlone_unsupported.icpp
View File

@ -18,6 +18,7 @@
#include "errno.hpp"
namespace fs
{
int

3
src/fs_file_size.cpp
View File

@ -18,7 +18,8 @@
#include "fs_fstat.hpp"
#include <stdint.h>
#include <cstdint>
namespace fs
{

3
src/fs_file_size.hpp
View File

@ -18,7 +18,8 @@
#pragma once
#include <stdint.h>
#include <cstdint>
namespace fs
{

1
src/fs_findallfiles.cpp
View File

@ -22,6 +22,7 @@
#include <string>
#include <vector>
namespace fs
{
void

9
src/fs_findallfiles.hpp
View File

@ -18,13 +18,16 @@
#pragma once
#include "strvec.hpp"
#include <string>
#include <vector>
namespace fs
{
void
findallfiles(const std::vector<std::string> &basepaths,
const char *fusepath,
std::vector<std::string> *paths);
findallfiles(const StrVec &basepaths,
const char *fusepath,
StrVec *paths);
}

32
src/fs_findonfs.cpp
View File

@ -16,7 +16,7 @@
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "branch.hpp"
#include "branches.hpp"
#include "errno.hpp"
#include "fs_fstat.hpp"
#include "fs_lstat.hpp"
@ -24,31 +24,29 @@
#include <string>
namespace l
{
static
int
findonfs(const BranchVec &branches_,
const std::string &fusepath_,
const int fd_,
std::string *basepath_)
findonfs(const Branches::CPtr &branches_,
const std::string &fusepath_,
const int fd_,
std::string *basepath_)
{
int rv;
dev_t dev;
struct stat st;
std::string fullpath;
const Branch *branch;
rv = fs::fstat(fd_,&st);
if(rv == -1)
return -1;
dev = st.st_dev;
for(size_t i = 0, ei = branches_.size(); i != ei; i++)
for(const auto &branch : *branches_)
{
branch = &branches_[i];
fullpath = fs::path::make(branch->path,fusepath_);
fullpath = fs::path::make(branch.path,fusepath_);
rv = fs::lstat(fullpath,&st);
if(rv == -1)
@ -57,7 +55,7 @@ namespace l
if(st.st_dev != dev)
continue;
*basepath_ = branch->path;
*basepath_ = branch.path;
return 0;
}
@ -69,13 +67,11 @@ namespace l
namespace fs
{
int
findonfs(const Branches &branches_,
const std::string &fusepath_,
const int fd_,
std::string *basepath_)
findonfs(const Branches::CPtr &branches_,
const std::string &fusepath_,
const int fd_,
std::string *basepath_)
{
rwlock::ReadGuard guard(branches_.lock);
return l::findonfs(branches_.vec,fusepath_,fd_,basepath_);
return l::findonfs(branches_,fusepath_,fd_,basepath_);
}
}

11
src/fs_findonfs.hpp
View File

@ -18,15 +18,16 @@
#pragma once
#include "branch.hpp"
#include "branches.hpp"
#include <string>
namespace fs
{
int
findonfs(const Branches &branches,
const std::string &fusepath,
const int fd,
std::string *basepath);
findonfs(const Branches::CPtr &branches,
const std::string &fusepath,
const int fd,
std::string *basepath);
}

1
src/fs_flistxattr.hpp
View File

@ -23,6 +23,7 @@
#include <sys/types.h>
namespace fs
{
static

1
src/fs_flock.hpp
View File

@ -20,6 +20,7 @@
#include <sys/file.h>
namespace fs
{
static

1
src/fs_fsetxattr.hpp
View File

@ -25,6 +25,7 @@
#include <sys/types.h>
namespace fs
{
static

1
src/fs_fstat.hpp
View File

@ -23,6 +23,7 @@
#include <sys/types.h>
#include <unistd.h>
namespace fs
{
static

1
src/fs_fstatat.hpp
View File

@ -23,6 +23,7 @@
#include <unistd.h>
#include <fcntl.h>
namespace fs
{
static

1
src/fs_fsync.hpp
View File

@ -20,6 +20,7 @@
#include <unistd.h>
namespace fs
{
static

1
src/fs_ftruncate.hpp
View File

@ -21,6 +21,7 @@
#include <sys/types.h>
#include <unistd.h>
namespace fs
{
static

1
src/fs_futimens.hpp
View File

@ -33,6 +33,7 @@
#include "fs_futimens_generic.hpp"
#endif
namespace fs
{
static

1
src/fs_futimens_freebsd_11.hpp
View File

@ -20,6 +20,7 @@
#include <sys/stat.h>
namespace fs
{
static

1
src/fs_futimens_generic.hpp
View File

@ -33,6 +33,7 @@
# define UTIME_OMIT ((1l << 30) - 2l)
#endif
namespace l
{
static

1
src/fs_futimens_linux.hpp
View File

@ -20,6 +20,7 @@
#include <sys/stat.h>
namespace fs
{
static

1
src/fs_futimesat.hpp
View File

@ -20,6 +20,7 @@
#include <sys/time.h>
namespace fs
{
int

1
src/fs_futimesat_generic.icpp
View File

@ -19,6 +19,7 @@
#include <fcntl.h>
#include <sys/time.h>
namespace fs
{
int

1
src/fs_futimesat_osx.icpp
View File

@ -26,6 +26,7 @@
#include <sys/time.h>
#include <unistd.h>
namespace l
{
static

1
src/fs_getdents64.cpp
View File

@ -23,6 +23,7 @@
#include <sys/syscall.h>
#endif
namespace fs
{
int

Some files were not shown because too many files have changed in this diff Show More