90d738b561
cache: implement rclone backend stats command
2020-04-29 10:10:57 +01:00
d80fdad6da
rc: implement backend/command for running backend commands remotely
2020-04-29 10:10:57 +01:00
e2916f3a55
local: implement backend command "noop" for testing purposes
2020-04-29 10:10:57 +01:00
1aa1a2c174
backend: add new backend command for backend specific commands
...
These commands are for implementing backend specific
functionality. They have documentation which is placed automatically
into the backend doc.
There is a simple test for the feature in the backend tests.
2020-04-29 10:10:57 +01:00
195d152785
rc: add GetStructMissingOK
2020-04-29 09:42:31 +01:00
1f61027f51
rc: add -o/--opt and -a/--arg for more structured input
2020-04-29 09:42:31 +01:00
37a53570d4
azureblob: implement memory pooling to control memory use
...
This commit implements memory pooling to control excessive memory use
as was implemented in the s3 backend.
2020-04-28 17:47:10 +01:00
ee7219aa20
azureblob: add --azureblob-disable-checksum flag
2020-04-28 17:47:10 +01:00
b1d8da484b
azureblob: retry InvalidBlobOrBlock error as it may indicate block concurrency problems
...
According to Microsoft support this error can be caused by
> A timing/concurrency issue where the PUT operations are happening
> about the same time for a single blob. The Put Block List operation
> writes a blob by specifying the list of block IDs that make up the
> blob. In order to be written as part of a blob, a block must have
> been successfully written to the server in a prior Put Block
> operation.
>
> Documentation reference:
>
> https://docs.microsoft.com/en-us/rest/api/storageservices/put-block
>
> This error can happen when doing concurrent upload commits after you
> have started the upload but before you commit. In that case, the
> upload fails. The application can retry this error or attempt some
> other recovery action based on the required scenario.
See: https://forum.rclone.org/t/error-while-syncing-with-azure-blob-storage-x-ms-error-code-invalidbloborblock/15561
2020-04-28 17:47:10 +01:00
4e869e03f7
s3: improve docs for --s3-disable-checksum
2020-04-28 17:47:10 +01:00
52c9647b06
b2: improve docs for --b2-disable-checksum
2020-04-28 17:47:10 +01:00
7238ae18f9
Add Adam Stroud to contributors
2020-04-28 17:47:10 +01:00
551a829eba
googlephotos: don't put an image in error message - fixes #4144
...
For a certain class of broken or missing image Google Photos puts an
image in the error message.
Before this fix we blindly chucked it into the error message.
After this fix we replace it with some sensible text.
2020-04-28 16:51:47 +01:00
8e91f83174
googlecloudstorage: Add ARCHIVE storage class to help
2020-04-27 11:40:21 +01:00
7f776c64f0
fichier: implement custom pacer to deal with the new rate limiting
2020-04-26 20:38:56 +02:00
8bf6ab2c52
accounting: fix race condition in tests
2020-04-24 12:32:09 +01:00
75fc3fe389
fs: fix FixRangeOption so it doesn't add HTTPOptions in place of bad Ranges
...
Before this fix, FixRangeOption would substitute RangeOptions it
wanted to get rid of with with empty HTTPOption. This caused a problem
now that backends interpret HTTPOptions.
This fix subsitutes those with NullOptions which aren't interpreted as
HTTPOptions. This patch also improves the unit tests.
2020-04-24 12:32:09 +01:00
c4572ebc91
rc: fix misplaced http server config - fixes #4130
2020-04-23 20:22:47 +01:00
e56976839a
jwtutil: Fix error handling
2020-04-23 17:52:14 +01:00
0c0ed2fe04
box: Remove unnecessary iat from jws claims
2020-04-23 17:52:14 +01:00
ab6ed256e5
putio: add support for --header-upload and --header-download #59
2020-04-23 15:55:52 +01:00
7c98ecd3ab
putio: make downloading files use the rclone http Client
...
This fixes `--download-header` and these transactions being missed from
`--dump bodies` or `--tpslimit`
2020-04-23 15:48:30 +01:00
f6346a4d29
fs: add --header flag to add options to every HTTP transaction #59
2020-04-23 15:24:21 +01:00
b502a74cff
gcs: add support for --header-upload and --header-download #59
2020-04-23 11:41:57 +01:00
8e9c25063a
swift: add support for --header-upload and --header-download #59
2020-04-23 11:34:36 +01:00
1dced3b3c4
rcat: add support for --header-upload #59
2020-04-23 11:34:31 +01:00
087bf1d584
cat: add support for --header-download #59
2020-04-23 11:34:24 +01:00
e051a34fc1
dbhashsum: deprecate: use rclone hashsum DropboxHash instead
2020-04-23 11:13:13 +01:00
f5455d865b
accounting: check for max transfer in WriteTo
...
Before this change the max transfer tests were failing for remotes
which were using WriterTo.
2020-04-23 11:13:13 +01:00
b705ead3fd
docs: adds --header-download and --header-upload
2020-04-23 11:07:21 +01:00
c390fc8100
onedrive: pass options to rest.Opts for Put and Update
2020-04-23 11:07:21 +01:00
14f6ce1e77
premiumizeme: pass options to rest.Opts for Put and Update
2020-04-23 11:07:21 +01:00
385542e2f9
sharefile: pass options to rest.Opts for Put and Update
2020-04-23 11:07:21 +01:00
fc946d0c44
fichier: pass options to rest.Opts for uploadFile
2020-04-23 11:07:21 +01:00
854c84d0ca
pcloud: pass options to rest.Opts for Put and Update
2020-04-23 11:07:21 +01:00
90bd0eb44c
webdav: pass options to rest.Opts for Put and Update
2020-04-23 11:07:21 +01:00
3130f870bb
sugarsync: pass options to rest.Opts for Put and Update
2020-04-23 11:07:21 +01:00
51b617f601
yandex: pass options to rest.Opts for upload
2020-04-23 11:07:21 +01:00
011ca244b2
jottacloud: pass options to rest.Opts for Put and Update
2020-04-23 11:07:21 +01:00
9ea1361044
googlephotos: pass options to rest.Opts for Put and Update
2020-04-23 11:07:21 +01:00
776966e22c
opendrive: pass options to rest.Opts for Put and Update
2020-04-23 11:07:21 +01:00
01cb256b84
box: pass options to rest.Opts for uploadPart
2020-04-23 11:07:21 +01:00
0b0163dde2
box: pass options to rest.Opts for upload
2020-04-23 11:07:21 +01:00
38123c70eb
b2: pass options to rest.Opts for Update
2020-04-23 11:07:21 +01:00
5cb7229a16
s3: add support for HTTPOption
2020-04-23 11:07:21 +01:00
9bf3d3da4c
fs: add UploadHeaders, DownloadHeaders to Update/Put/Open options
2020-04-23 11:07:21 +01:00
93caa459e3
fs/config: add header-download and header-upload flags
2020-04-23 11:07:21 +01:00
f8039deb7c
s3: fix detection of BucketAlreadyOwnedByYou and BucketAlreadyExists error
...
This was being silently ignored until this commit
e2bf91452a263ff8 s3: report errors on bucket creation (mkdir) correctly
2020-04-22 18:14:03 +01:00
86eaf43b00
vfs: fix tests for Statfs when running on backends with unknowns
...
This was broken in da41db4712383027
2020-04-22 18:14:03 +01:00
8176202e6d
Add Sunil Patra to contributors
2020-04-22 18:14:03 +01:00
