Commit Graph

8380 Commits

Author SHA1 Message Date
156feff9f2 sftp: support listening on passed FDs 2024-09-06 17:21:56 +01:00
b29a22095f http: fix addr CLI arg help text
This was missing the fact rclone also supports listening on Unix Domain
Sockets.
2024-09-06 17:21:56 +01:00
861c01caf5 http: support listening on passed FDs
Instead of the listening addresses specified above, rclone will listen to all
FDs passed by the service manager, if any (and ignore any arguments passed by
`--{{ .Prefix }}addr`.

This allows rclone to be a socket-activated service. It can be configured as described in
https://www.freedesktop.org/software/systemd/man/latest/systemd.socket.html

It's possible to test this interactively through `systemd-socket-activate`,
firing of a request in a second terminal:

```
❯ systemd-socket-activate -l 8088 -l 8089 --fdname=foo:bar -- ./rclone serve webdav :local:test/
Listening on [::]:8088 as 3.
Listening on [::]:8089 as 4.
Communication attempt on fd 3.
Execing ./rclone (./rclone serve webdav :local:test/)
2024/04/24 18:14:42 NOTICE: Local file system at /home/flokli/dev/flokli/rclone/test: WebDav Server started on [sd-listen:bar-0/ sd-listen:foo-0/]
```
2024-09-06 17:21:56 +01:00
f1a84d171e build: fix build after update
This adds an import missed in

bcdfad3c83720c11 build: update logging statements to make json log work #6038
2024-09-06 17:18:46 +01:00
bcdfad3c83 build: update logging statements to make json log work - fixes #6038
This changes log statements from log to fs package, which is required for --use-json-log
to properly make log output in JSON format. The recently added custom linting rule,
handled by ruleguard via gocritic via golangci-lint, warns about these and suggests
the alternative. Fixing was therefore basically running "golangci-lint run --fix",
although some manual fixup of mainly imports are necessary following that.
2024-09-06 17:04:18 +01:00
88b0757288 build: update custom linting rule for log to suggest new non-format functions 2024-09-06 17:04:18 +01:00
33d6c3f92f fs: add non-format variants of log functions to avoid non-constant format string warnings 2024-09-06 17:04:18 +01:00
752809309d fs: add log Printf, Fatalf and Panicf 2024-09-06 17:04:18 +01:00
4a54cc134f fs: refactor base log method name for improved consistency 2024-09-06 17:04:18 +01:00
dfc2c98bbf fs: refactor log statements to use common helper 2024-09-06 17:04:18 +01:00
604d6bcb9c build: enable custom linting rules with ruleguard via gocritic 2024-09-06 17:04:18 +01:00
d15704ef9f rcserver: implement prometheus metrics on a dedicated port - fixes #7940 2024-09-06 15:00:36 +01:00
26bc9826e5 swift: add total/free space info in about command.
With the enhancement in version v2.0.3 of ncw/swift library, we can now get Total and Free space info from remotes that support this feature (ex. Blomp storage)
2024-09-06 12:46:51 +01:00
2a28b0eaf0 docs: filtering: added Byte unit for min/max-size parameters. 2024-09-06 12:28:29 +01:00
2d1c2b1f76 config encryption: set, remove and check to manage config file encryption #7859
Some checks are pending
Docker beta build / Build image job (push) Waiting to run
2024-09-06 10:34:29 +01:00
ffb2e2a6de config: use --password-command to set config file password if supplied
Before this change, rclone ignored the --password-command on the
rclone config setting except when decrypting an existing config file.

This change allows for offloading the password storage/generation into
external hardware key or other protected password storage.

Fixes #7859
2024-09-06 10:34:29 +01:00
c9c283533c config: factor --password-command code into its own function #7859 2024-09-06 10:34:29 +01:00
71799d7efd Add yuval-cloudinary to contributors 2024-09-06 10:34:29 +01:00
8f4fdf6cc8 Add nipil to contributors 2024-09-06 10:34:29 +01:00
91b11f9eac documentation: add cheatsheet for configuration encryption
Some checks are pending
Docker beta build / Build image job (push) Waiting to run
2024-09-05 17:39:25 +01:00
b49927fbd0 docs: more secure two-step signature and hash validation 2024-09-05 16:54:26 +01:00
1a8b7662e7 serve nfs: unify the nfs library logging with rclone's logging better
Some checks failed
Docker beta build / Build image job (push) Has been cancelled
Before this we ignored the logging levels and logged everything as
debug. This will obey the rclone logging flags and log at the correct
level.
2024-09-04 10:50:21 +01:00
6ba3e24853 serve nfs: fix incorrect user id and group id exported to NFS #7973
Before this change all exports were exported as root and the --uid and
--gid flags of the VFS were ignored.

This fixes the issue by exporting the UID and GID correctly which
default to the current user and group unless set explicitly.
2024-09-04 10:50:21 +01:00
802a938bd1 zoho: fix inefficiencies uploading with new API to avoid throttling
Before this fix, rclone queried the uploaded object to find its size
and modtime after upload as the API did not return these items.

Zoho have subsequently modified the API to return these items so
rclone uses them to avoid an API call.

This should help with rclone being throttled by Zoho.

See: https://forum.rclone.org/t/second-followup-on-the-older-topic-rclone-invokes-more-number-of-workdrive-s-files-listing-api-calls-which-exceeds-the-throttling-limit/45697/20
2024-09-04 10:45:47 +01:00
9deb3e8adf Add crystalstall to contributors 2024-09-04 10:45:12 +01:00
296281a6eb docs: fix some function names in comments
Some checks failed
Docker beta build / Build image job (push) Has been cancelled
Signed-off-by: crystalstall <crystalruby@qq.com>
2024-09-02 18:20:08 +02:00
711478554e lib/file: use builtin MkdirAll with go1.22 instead of our own custom version for windows
Starting with go1.22 the standard os.MkdirAll has improved its handling of volume names,
and as part of that it now stops recursing into parent directory if it is a volume name
(see: cd589c8a73).
This is similar to what was our main change and reason for creating a custom version. When
building with go1.22 or newer we can therefore stop using our custom version, with the
advantage that we automatically get current and future relevant improvements from golang.
To support building with go1.21 the existing custom version is still kept, and therefore
also our wrapper function file.MkdirAll - but it now just calls os.MkdirAll with go1.22
or newer on Windows.

See #5401, #6420 and acf1e2df84a350b7a86d7672d749dfb1ba090a44 for details about the
creation of our custom version of MkdirAll.
2024-09-02 18:16:38 +02:00
906aef91fa docs: document that paths using volume guids are supported 2024-09-02 18:01:47 +02:00
6b58cd0870 s3: fix accounting for mulpart transfers after migration to SDKv2 #4989
Some checks failed
Docker beta build / Build image job (push) Has been cancelled
2024-08-31 20:11:53 +01:00
af9f8ced80 yandex: implement custom user agent to help with upload speeds
Some checks failed
Docker beta build / Build image job (push) Has been cancelled
2024-08-29 18:25:08 +01:00
c63f1865f3 operations: copy: generate stable partial suffix
Some checks failed
Docker beta build / Build image job (push) Has been cancelled
2024-08-28 08:45:38 +02:00
1bb89bc818 docs: add missing sftp providers to README and main docs page - fixes #8038 2024-08-28 07:25:49 +01:00
a365503750 nfsmount: fix stale handle problem after converting options to new style
This problem was caused by the defaults not being set for the options
after the conversion to the new config system in

28ba4b832d009133 serve nfs: convert options to new style

This makes the nfs serve options globally available so nfsmount can
use them directly.

Fixes #8029
2024-08-28 07:08:23 +01:00
3bb6d0a42b docs: mark flags.md as auto generated so contributors don't edit it 2024-08-28 07:08:23 +01:00
f65755b3a3 Add Pawel Palucha to contributors 2024-08-28 07:08:21 +01:00
33c5f35935 Add John Oxley to contributors 2024-08-28 07:07:34 +01:00
4367b999c9 Add Georg Welzel to contributors 2024-08-28 07:04:02 +01:00
b57e6213aa Add Péter Bozsó to contributors 2024-08-28 07:04:02 +01:00
cd90ba4337 Add Sam Harrison to contributors 2024-08-28 07:04:02 +01:00
0e5eb7a9bb s3: allow restoring from intelligent-tiering storage class
Some checks failed
Docker beta build / Build image job (push) Has been cancelled
2024-08-25 15:08:06 +02:00
956c2963fd bisync: don't convert modtime precision in listings - fixes #8025
Some checks are pending
Docker beta build / Build image job (push) Waiting to run
Before this change, bisync proactively converted modtime precision when greater
than what the destination backend supported.

This dates back to a time before bisync considered the modifyWindow for same-side
comparisons. Back then, it was problematic to save a listing with 12:54:49.7 for
a backend that can't handle that precision, as on the next run the backend would
report the time as 12:54:50 and bisync would think the file had changed. So the
truncation was a workaround to anticipate this and proactively record the time
with the precision we expect to receive next time.

However, this caused problems for backends (such as dropbox) that round instead
of truncating as bisync expected.

After this change, bisync preserves the original precision in the listing
(without conversion), even when greater than what the backend supports, to avoid
rounding error. On the next run, bisync will compare it to the rounded time
reported by the backend, and if it's within the modifyWindow, it will treat them
as equivalent.
2024-08-24 22:32:48 -04:00
146562975b build: rename Unknwon/goconfig to unknwon/goconfig
Some checks failed
Docker beta build / Build image job (push) Has been cancelled
Before this change we used the repo with an initial uppercase `U`. However it is now canonically spelled with a lower case `u`.

This package is too old to have a go.mod but the README clearly states the desired capitalization.

In 4b0d4b818a the
recommended capitalization was changed to lower case.

Co-authored-by: John Oxley <joxley@meta.com>
2024-08-23 11:03:27 +01:00
4c1cb0622e backend: pcloud: Implement OpenWriterAt feature
Some checks are pending
Docker beta build / Build image job (push) Waiting to run
2024-08-22 23:42:32 +02:00
258092f9c6 backend: pcloud: implement SetModTime - Fixes #7896 2024-08-22 23:42:32 +02:00
be448c9e13 filescom: don't make an extra fetch call on each item in a list response
Some checks failed
Docker beta build / Build image job (push) Has been cancelled
2024-08-19 22:31:57 +02:00
4e708e59f2 local: fix incorrect conversion between integer types
Some checks failed
Docker beta build / Build image job (push) Has been cancelled
2024-08-18 10:29:36 +02:00
c8366dfef3 local: fix incorrect conversion between integer types
Some checks are pending
Docker beta build / Build image job (push) Waiting to run
2024-08-17 17:07:17 +02:00
1e14523b82 docs: make tardigrade page auto redirect to storj page 2024-08-17 16:00:42 +02:00
da25305ba0 docs: update backend config samples 2024-08-17 16:00:18 +02:00
e439121ab2 config: fix size computation for allocation may overflow 2024-08-17 15:03:39 +02:00