Commit Graph

7396 Commits

Author SHA1 Message Date
dc803b572c Add hideo aoyama to contributors 2023-08-18 12:05:15 +01:00
4d19042a61 Add Jacob Hands to contributors 2023-08-18 12:05:15 +01:00
923989d1d7 build: add snap installation
I ( @boukendesho ) have volunteered to maintain the snap package so
this adds it back into the installation instructions.

It will set a `snap` tag visible in `rclone version` so we know where
it came from for support queries.
2023-08-18 11:57:25 +01:00
cf65e36cf3 ci: change Winget Releaser job to ubuntu-latest 2023-08-17 11:36:28 +01:00
cf5457c2cd fs: Fix transferTime not being set in JSON logs
This was unintentionally broken in 04aa696
2023-08-17 11:19:35 +01:00
ea4aa696a5 fs: Don't stop calculating average transfer speed until the operation is complete
Currently, the average transfer speed will stop calculating 1 minute
after the last queued transfer completes. This causes the average to
stop calculating when checking is slow and the transfer queue becomes
empty.

This change will require all checks to complete before stopping the
average speed calculation.
2023-08-16 21:43:24 +01:00
34195fd3e8 sync: fix erroneous test in TestSyncOverlapWithFilter
In this commit:

432d5d1e20abe644 operations: fix overlapping check on case insensitive file systems

We introduced a test that makes no sense. This happens to pass without --fast-list and fail with it.

This removes the test.
2023-08-13 11:42:31 +01:00
40b8167ab4 Add Vitor Gomes to contributors 2023-08-13 11:42:22 +01:00
e365f237f5 Add nielash to contributors 2023-08-13 11:41:24 +01:00
7d449572bd Add alexia to contributors 2023-08-13 11:41:24 +01:00
181fecaec3 multithread: refactor multithread operation to use OpenChunkWriter if available #7056
If the feature OpenChunkWriter is not available, multithread tries to create an adapter from OpenWriterAt to OpenChunkWriter.
2023-08-12 17:55:01 +01:00
7701d1d33d config: add "multi-thread-chunk-size" flag #7056 2023-08-12 17:55:01 +01:00
6dd736fbdc s3: refactor MultipartUpload to use OpenChunkWriter and ChunkWriter #7056 2023-08-12 17:55:01 +01:00
f36ca0cd25 features: add new interfaces OpenChunkWriter and ChunkWriter #7056 2023-08-12 17:55:01 +01:00
9b3b1c7067 bisync: typo corrections & other doc improvements 2023-08-12 17:24:21 +01:00
0dd0d6a13e bisync: Add support for --create-empty-src-dirs - Fixes #6109
Sync creation and deletion of empty directories.
https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=3.%20Bisync%20should%20create/delete%20empty%20directories%20as%20sync%20does%2C%20when%20%2D%2Dcreate%2Dempty%2Dsrc%2Ddirs%20is%20passed

Also fixed an issue causing --resync to erroneously delete empty folders and duplicate files unique to Path2
https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=2.%20%2D%2Dresync%20deletes%20data%2C%20contrary%20to%20docs
2023-08-12 17:24:21 +01:00
e5bde42303 bisync: Add experimental --resilient mode to allow recovery from self-correctable errors
https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=2.%20Bisync%20should%20be%20more%20resilient%20to%20self%2Dcorrectable%20errors
2023-08-12 17:24:21 +01:00
f01a50eb47 bisync: Add new --ignore-listing-checksum flag to distinguish from --ignore-checksum
https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=6.%20%2D%2Dignore%2Dchecksum%20should%20be%20split%20into%20two%20flags%20for%20separate%20purposes
2023-08-12 17:24:21 +01:00
5ca61ab705 bisync: equality check before renaming (leave identical files alone)
Improved detection of false positive change conflicts (identical files are now left alone instead of renamed)
https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=1.%20Identical%20files%20should%20be%20left%20alone%2C%20even%20if%20new/newer/changed%20on%20both%20sides
2023-08-12 17:24:21 +01:00
4ac4ce6afd bisync: apply filters correctly during deletes
Fixed an issue causing bisync to consider more files than necessary due to overbroad filters during delete operations
https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=5.%20Bisync%20reads%20files%20in%20excluded%20directories%20during%20delete%20operations
2023-08-12 17:24:21 +01:00
40a874a0d8 bisync: enforce --check-access during --resync
--check-access is now enforced during --resync, preventing data loss in certain user error scenarios
https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=%2D%2Dcheck%2Daccess%20doesn%27t%20always%20fail%20when%20it%20should
2023-08-12 17:24:21 +01:00
f4dd86238d bisync: dry runs no longer commit filter changes
Fixed an issue causing dry runs to inadvertently commit filter changes
https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=1.%20Dry%20runs%20are%20not%20completely%20dry
2023-08-12 17:24:21 +01:00
4f1eafb044 gitignore: add .DS_Store and remove *.log
.DS_Store is an irrelevant file created automatically by macOS.

*.log can't be ignored because bisync uses .log files in its test suite.
2023-08-12 17:24:21 +01:00
20c9e0cab6 fichier: fix error code parsing
This fixes the following error I encountered:

```
2023/08/09 16:18:49 DEBUG : failed parsing fichier error: strconv.Atoi: parsing "#374": invalid syntax
2023/08/09 16:18:49 DEBUG : pacer: low level retry 1/10 (error HTTP error 403 (403 Forbidden) returned body: "{\"status\":\"KO\",\"message\":\"Flood detected: IP Locked #374\"}")
```
2023-08-11 00:47:01 +09:00
9c09cf9cf6 build: update to released go1.21 2023-08-09 22:41:19 +01:00
3a3af00180 Add antoinetran to contributors 2023-08-09 22:41:19 +01:00
281e0c2d62 Add James Braza to contributors 2023-08-09 22:41:19 +01:00
25b81b8789 Add Masamune3210 to contributors 2023-08-09 22:41:19 +01:00
90fdd97a7b Add Nihaal Sangha to contributors 2023-08-09 22:41:19 +01:00
3c58e0efe0 protondrive: update the information regarding the advance setting enable_caching (#7202) 2023-08-09 16:01:19 +02:00
db744f64f6 docs: clarify --checksum documentation - Fixes #7145 2023-08-09 20:00:46 +09:00
480220a84a docs: add some more docs on making your own backend 2023-08-09 20:00:46 +09:00
d0362171cf docs: environment variable remote name only supports letters, digits, or underscores 2023-08-09 11:42:04 +01:00
45887d11f6 docs: local: fix typo 2023-08-09 11:32:45 +01:00
c4bad5c1bc lib/rest: remove unnecessary nil check
From the Go docs:

  "A `nil` map is equivalent to an empty map. [1]

Therefore, an additional nil check for `opts.ExtraHeaders` before the loop is
unnecessary because `opts.ExtraHeaders` is a `map`.

[1]: https://go.dev/ref/spec#Map_types

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2023-08-09 19:17:42 +09:00
40de89df73 drive: fix typo in docs 2023-08-05 12:51:51 +01:00
27f5297e8d oracleobjectstorage: Use rclone's rate limiter in mutipart transfers 2023-08-05 12:09:17 +09:00
de185de215 accounting: show server side stats in own lines and not as bytes transferred
Before this change we showed both server side moves and server side
copies as bytes transferred.

This made a nice easy to use stats display, but also caused confusion
for users who saw unrealistic transfer times. It also caused a problem
with --max-transfer and chunker which renames each chunk after
uploading which was counted as a transfer byte.

This patch instead accounts the server side move and copy statistics
as a seperate lines in the stats display which will only appear if
there are any server side moves / copies. This is also output in the
rc.

This gives users something to look at when transfers are running which
was the point of the original change but it now means that transfer
bytes represents data transfers through this rclone instance only.

Fixes #7183
2023-08-05 03:54:01 +01:00
d362db2e08 rclone test info: add --check-base32768 flag to check can store all base32768 characters
Fixes #7208
2023-08-05 02:59:58 +01:00
db2a49e384 Add Raymond Berger to contributors 2023-08-05 02:59:58 +01:00
d63fcc6e44 storj: performance improvement for large file uploads
storj.io/uplink v1.11.0 comes with an improved logic for uploading large
files where file segments are uploaded concurrently instead of serially.
This allows to fully utilize the network connection during the entire
upload process.

This change enable the new upload logic.
2023-08-04 17:40:03 +09:00
4444037f5c docs: box client_id creation
See: https://forum.rclone.org/t/box-getting-403-errors-when-using-chunker-working-fine-without-it/40388/22
2023-08-03 14:51:49 +01:00
a555513c26 docs: add missing comma to overview webdav footnote 2023-08-03 14:50:04 +01:00
039c260216 build: update to go1.21rc4 2023-08-03 13:53:43 +01:00
4577c08e05 Add Julian Lepinski to contributors 2023-08-03 13:53:43 +01:00
c9ed691919 docs: add minio as a sponsor 2023-08-03 08:39:15 +01:00
9f96c0d4ea swift: fix HEADing 0-length objects when --swift-no-large-objects set
The Swift backend does not always respect the flag telling it to skip
HEADing zero-length objects. This commit fixes that for ls/lsl/lsf.

Swift returns zero length for dynamic large object files when they're
included in a files lookup, which means that determining their size
requires HEADing each file that returns a size of zero. rclone's
--swift-no-large-objects instructs rclone that no large objects are
present and accordingly rclone should not HEAD files that return zero
length.

When rclone is performing an ls / lsf / lsl type lookup, however, it
continues to HEAD any zero length objects it encounters, even with
this flag set. Accordingly, this change causes rclone to respect the
flag in these situations.

NB: It is worth noting that this will cause rclone to incorrectly
report zero length for any dynamic large objects encountered with the
--swift-no-large-objects flag set.
2023-08-03 08:38:39 +01:00
91d095f468 docs: update command docs to new style 2023-08-02 12:53:09 +01:00
bff702a6f1 docs: group the global flags and make them appear on command and flags pages
This adds an additional parameter to the creation of each flag. This
specifies one or more flag groups. This **must** be set for global
flags and **must not** be set for local flags.

This causes flags.md to be built with sections to aid comprehension
and it causes the documentation pages for each command (and the
`--help`) to be built showing the flags groups as specified in the
`groups` annotation on the command.

See: https://forum.rclone.org/t/make-docs-for-mortals-not-only-rclone-gurus/39476/
2023-08-02 12:53:09 +01:00
a1d6bbd31f Add rclone completion powershell - basic implementation only 2023-08-02 12:53:09 +01:00