8539 Commits

Author SHA1 Message Date
Nick Craig-Wood
4e77a4ff73 Version v1.69.1 v1.69.1 2025-02-14 15:17:21 +00:00
Anagh Kumar Baranwal
b63c42f39b build: disable docker builds on PRs & add missing dockerfile changes
Signed-off-by: Anagh Kumar Baranwal <6824881+darthShadow@users.noreply.github.com>
2025-02-14 13:03:33 +00:00
Anagh Kumar Baranwal
30c9bab35d Added parallel docker builds and caching for go build in the container
Signed-off-by: Anagh Kumar Baranwal <6824881+darthShadow@users.noreply.github.com>
2025-02-14 13:03:33 +00:00
Nick Craig-Wood
68bbd8017d docs: add FileLu as sponsors and tidy sponsor logos 2025-02-14 13:03:33 +00:00
Nick Craig-Wood
259dbbab55 vfs: fix the cache failing to upload symlinks when --links was specified
Before this change, if --vfs-cache-mode writes or above was set and
--links was in use, when a symlink was saved then the VFS failed to
upload it. This meant when the VFS was restarted the link wasn't there
any more.

This was caused by the local backend, which we use to manage the VFS
cache, picking up the global --links flag.

This patch makes sure that the internal instantations of the local
backend in the VFS cache don't ever use the --links flag or the
--local-links flag even if specified on the command line.

Fixes #8367
2025-02-14 13:03:33 +00:00
Zachary Vorhies
5fa85f66fe doc: add note on concurrency of rclone purge 2025-02-14 13:03:33 +00:00
jbagwell-akamai
fb648e4774 s3: add latest Linode Object Storage endpoints
Added missing Linode Object Storage endpoints AMS, MAA, CGK, LON, LAX, MAD, MEL, MIA, OSA, GRU, SIN
2025-02-14 13:03:33 +00:00
nielash
9978750a8c fix golangci-lint errors 2025-02-14 13:03:33 +00:00
nielash
d953c0c51b bisync: fix listings missing concurrent modifications - fixes #8359
Before this change, there was a bug affecting listing files when:

- a given bisync run had changes in the 2to1 direction
AND
- the run had NO changes in the 1to2 direction
AND
- at least one of the changed files changed AGAIN during the run
(specifically, after the initial march and before the transfers.)

In this situation, the listings on one side would still retain the prior version
of the changed file, potentially causing conflicts or errors.

This change fixes the issue by making sure that if we're updating the listings
on one side, we must also update the other. (We previously tried to skip it for
efficiency, but this failed to account for the possibility that a changed file
could change again during the run.)
2025-02-14 13:03:33 +00:00
Nick Craig-Wood
9dfce11c9b lib/oauthutil: fix redirect URL mismatch errors - fixes #8351
In this commit we introduced support for client credentials flow:

65012beea4b93b7f lib/oauthutil: add support for OAuth client credential flow

This involved re-organising the oauth credentials.

Unfortunately a small error was made which used a fixed redirect URL
rather than the one configured for the backend.

This caused the box backend oauth flow not to work properly with
redirect_uri_mismatch errors.

These backends were using the wrong redirect URL and will likely be
affected, though it is possible the backends have workarounds.

- box
- drive
- googlecloudstorage
- googlephotos
- hidrive
- pikpak
- premiumizeme
- sharefile
- yandex
2025-02-14 13:03:33 +00:00
Nick Craig-Wood
504f2fb571 b2: fix "fatal error: concurrent map writes" - fixes #8355
This was caused by the embryonic metadata support. Since this isn't
actually visible externally, this patch removes it for the time being.
2025-02-14 13:03:33 +00:00
Zachary Vorhies
f79f929e57 serve nfs: update docs to note Windows is not supported - fixes #8352 2025-02-14 13:03:33 +00:00
jkpe
83e04ead37 s3: add DigitalOcean regions SFO2, LON1, TOR1, BLR1 2025-02-14 13:03:33 +00:00
Nick Craig-Wood
1a95a23fdc onedrive: mark German (de) region as deprecated
See: https://learn.microsoft.com/en-us/previous-versions/azure/germany/
2025-02-14 13:03:33 +00:00
Bruno Fernandes
c4b592e549 s3: Added new storage class to magalu provider 2025-02-14 13:03:33 +00:00
izouxv
642d1415d1 vfs: close the change notify channel on Shutdown 2025-02-14 13:03:33 +00:00
Tim White
64556d4ca2 docs: add OneDrive Impersonate instructions - fixes #5610 2025-02-14 13:03:33 +00:00
Christoph Berger
de69448565 docs: explain the stringArray flag parameter descriptor 2025-02-14 13:03:33 +00:00
Nick Craig-Wood
ad941655c5 iclouddrive: add notes on ADP and Missing PCS cookies - fixes #8310 2025-02-14 13:03:33 +00:00
Dimitri Papadopoulos
6cbb9fd7cb docs: fix typos found by codespell in docs and code comments 2025-02-14 13:03:33 +00:00
Nick Craig-Wood
7988300f50 fs: fix confusing "didn't find section in config file" error
This change decorates the error with the section name not found which
will hopefully save user confusion.

Fixes #8170
2025-02-14 13:03:33 +00:00
Nick Craig-Wood
1b47b7a6bb vfs: fix race detected by race detector
This race would only happen when --dir-cache-time was very small.

This was noticed in the VFS tests when --dir-cache-time was 100 mS so
is unlikely to affect normal users.
2025-02-14 13:03:33 +00:00
Matt Ickstadt
626bdacd59 docs: fix reference to serves3 setting disable_multipart_uploads which was renamed 2025-02-14 13:03:33 +00:00
Matt Ickstadt
1ef2da31a7 docs: fix link to Rclone Serve S3 2025-02-14 13:03:33 +00:00
Nick Craig-Wood
376a5b1a83 serve s3: fix list objects encoding-type
Before this change rclone would always use encoding-type url even if
the client hadn't asked for it.

This confused some clients.

This fixes the problem by leaving the URL encoding to the gofakes3
library which has also been fixed.

Fixes #7836
2025-02-14 13:03:33 +00:00
Nick Craig-Wood
ddaeb07019 doc: make man page well formed for whatis - fixes #7430 2025-02-14 13:03:33 +00:00
Nick Craig-Wood
c72f71bd02 Start v1.69.1-DEV development 2025-02-14 13:03:33 +00:00
Nick Craig-Wood
cd3b08d8cf Version v1.69.0 v1.69.0 2025-01-12 15:09:13 +00:00
Nick Craig-Wood
009660a489 test_all: disable docker plugin tests
These are not completing on the integration test server. This needs
investigating, but we need the integration tests to run properly.
2025-01-12 14:02:57 +00:00
albertony
4b6c7c6d84 docs: fix typo 2025-01-12 13:49:47 +01:00
Nick Craig-Wood
a7db375f5d accounting: fix race stopping/starting the stats counter
Some checks are pending
Docker beta build / Build image job (push) Waiting to run
This was picked up by the race detector in the CI.
2025-01-11 20:25:34 +00:00
Nick Craig-Wood
101dcfe157 docs: add github.com/icholy/gomajor to RELEASE for updating major versions 2025-01-11 20:25:34 +00:00
Francesco Frassinelli
aec87b74d3
ftp: fix ls commands returning empty on "Microsoft FTP Service" servers
The problem was in the upstream library jlaffaye/ftp and this updates it.

Fixes #8224
2025-01-11 20:02:16 +00:00
Nick Craig-Wood
91c8f92ccb s3: add docs on data integrity
Some checks are pending
Docker beta build / Build image job (push) Waiting to run
See: https://forum.rclone.org/t/help-me-figure-out-how-to-verify-backup-accuracy-and-completeness-on-s3/37632/5
2025-01-11 18:39:15 +00:00
Nick Craig-Wood
965bf19065 webdav: make --webdav-auth-redirect to fix 401 unauthorized on redirect
Before this change, if the server returned a 302 redirect message when
opening a file rclone would do the redirect but drop the
Authorization: header. This is a sensible thing to do for security
reasons but breaks some setups.

This patch adds the --webdav-auth-redirect flag which makes it
preserve the auth just for this kind of request.

See: https://forum.rclone.org/t/webdav-401-unauthorized-when-server-redirects-to-another-domain/39292
2025-01-11 18:39:15 +00:00
Nick Craig-Wood
15ef3b90fa rest: make auth preserving redirects an option 2025-01-11 18:39:15 +00:00
Nick Craig-Wood
f6efaf2a63 box: fix panic when decoding corrupted PEM from JWT file
See: https://forum.rclone.org/t/box-jwt-config-erroring-panic/40685/
2025-01-11 18:39:15 +00:00
Nick Craig-Wood
0e7c495395 size: make output compatible with -P
Before this change the output of `rclone size -P` would get corrupted
by the progress printing.

This is fixed by using operations.SyncPrintf instead of fmt.Printf.

Fixes #7912
2025-01-11 18:39:15 +00:00
Nick Craig-Wood
ff0ded8f11 vfs: add remote name to vfs cache log messages - fixes #7952 2025-01-11 18:39:15 +00:00
Nick Craig-Wood
110bf468a4 dropbox: fix return status when full to be fatal error
This will stop the sync, but won't stop a mount.

Fixes #7334
2025-01-11 18:39:15 +00:00
Nick Craig-Wood
d4e86f4d8b rc: add relative to vfs/queue-set-expiry 2025-01-11 18:39:15 +00:00
Nick Craig-Wood
6091a0362b vfs: fix open files disappearing from directory listings
In this commit

aaadb48d486cc880 vfs: keep virtual directory status accurate and reduce deadlock potential

We reworked the virtual directory detection to use an atomic bool so
that we could run part of the cache forgetting only with a read lock.

Unfortunately this had a bug which meant that directories with virtual
items could be forgotten.

This commit changes the boolean into a count of virtual entries which
should be more accurate.

Fixes #8082
2025-01-11 18:39:15 +00:00
Nick Craig-Wood
33d2747829 docker serve: parse all remaining mount and VFS options
Before this change, this code implemented an ad-hoc parser for a
subset of vfs and mount options.

After the config re-organization it can use the same parsing code as
the rest of rclone which simplifies the code and exposes all the VFS
and mount options.
2025-01-11 18:39:15 +00:00
Nick Craig-Wood
c9e5f45d73 smb: fix panic if stat fails
Before this fix the smb backend could panic if a stat call failed.

This fix makes it return an error instead.

It should have the side effect that we do one less stat call on upload
too.

Fixes #8106
2025-01-11 18:39:15 +00:00
Nick Craig-Wood
2f66537514 googlephotos: fix nil pointer crash on upload - fixes #8233 2025-01-11 18:39:15 +00:00
Nick Craig-Wood
a491312c7d iclouddrive: tweak docs 2025-01-11 18:39:15 +00:00
Nick Craig-Wood
45b7690867 serve dlna: sort the directory entries by directories first then alphabetically by name
Some media boxes don't sort the items returned from the DLNA server,
so sort them here, directories first then alphabetically by name.

See: https://forum.rclone.org/t/serve-dlna-files-order-directories-first/47790
2025-01-11 17:11:40 +00:00
Nick Craig-Wood
30ef1ddb23 serve nfs: fix missing inode numbers which was messing up ls -laR
Some checks failed
Docker beta build / Build image job (push) Has been cancelled
In 6ba3e2485399a918

    serve nfs: fix incorrect user id and group id exported to NFS #7973

We updated the stat function to output uid and gid. However this set
the inode numbers of everything to -1. This causes a problem with
doing `ls -laR` giving "not listing already-listed directory" as it
uses inode numbers to see if it has listed a directory or not.

This patch reads the inode number from the vfs.Node and sets it in the
Stat output.
2025-01-09 18:55:18 +00:00
Nick Craig-Wood
424d8e3123 serve nfs: implement --nfs-cache-type symlink
`--nfs-cache-type symlink` is similar to `--nfs-cache-type disk` in
that it uses an on disk cache, but the cache entries are held as
symlinks. Rclone will use the handle of the underlying file as the NFS
handle which improves performance.
2025-01-09 18:55:18 +00:00
Nick Craig-Wood
04dfa6d923 azureblob,oracleobjectstorage,s3: quit multipart uploads if the context is cancelled
Before this change the multipart uploads would continue retrying even
if the context was cancelled.
2025-01-09 18:55:18 +00:00