0a922ad1dc
acd: Reauth on 401 errors
...
Fixes #493
Fixes #501
2016-05-28 16:49:26 +01:00
536526cc92
amazonclouddrive: Restart directory listings on error - fixes #475
...
Before this change rclone would retry only the page that was missing
from the directory listing. However it turns out that on 429 errors
at least, that page is gone from the directory listing which results
in missing files in the list. The workaround for this is to restart
the directory listing on any retryable errors.
2016-05-14 17:15:42 +01:00
68ec6a9f5b
Add a directory parameter to Fs.List()
2016-05-06 16:52:34 +01:00
753b0717be
Refactor the List and ListDir interface
...
Gives more accurate error propagation, control of depth of recursion
and short circuit recursion where possible.
Most of the the heavy lifting is done in the "fs" package, making file
system implementations a bit simpler.
This commit contains some code originally by Klaus Post.
Fixes #316
2016-05-06 16:52:34 +01:00
1752ee3c8b
Retry errors which indicate the connection closed prematurely.
...
See discussion in #442
2016-04-29 17:29:34 +01:00
414b35ea56
Change the interface of SetModTime to return an error - #348
2016-03-22 15:56:44 +00:00
7d4e143dee
Make it obvious that the client secrets are encrypted
2016-02-28 19:57:19 +00:00
e08e35984c
Add help to remote chooser in rclone config - fixes #43
2016-02-21 13:40:16 +00:00
ef06371c93
Create separate interface for object information.
...
Take out read-only information about a Fs in a separate struct to limit access.
See discussion at #282 .
2016-02-19 13:31:09 +00:00
8cd3c25b41
Amazon Cloud Drive: retry on 400, 401, 408, 504 and EOF errors - fixes #340
2016-02-16 14:45:22 +00:00
d4df3f2154
acd: Download files >= 9GB with their tempLink direct from s3
...
This files the problem downloading files > 10GB.
Fixes #204 Fixes #313
2016-01-30 18:08:44 +00:00
c47b4f828f
acd: Fix deadlock in directory traversal code
2016-01-24 11:20:55 +00:00
83f0d3e03d
acd: remove 409 conflict from error codes we will retry
...
This should fix the very long pauses or getting stuck people have seen
in uploads.
2016-01-23 17:02:09 +00:00
ce305321b6
amazon cloud drive: Fix "Next token is expired" - Fixes #289 Fixes #263
...
This should also fix the consequent "409 Conflict" name already exists errors.
2016-01-20 20:05:52 +00:00
78c3a5ccfa
Add support for multiple hash types.
...
Add support for multiple hash types with negotiation of common hash types for comparison.
Manually rebased version of #277 (see discussion there)
2016-01-11 13:39:33 +01:00
5189231a34
Tweaks to rclone authorize
...
* Document the headless / remote setup procedure
* Move Config constants into fs
* Parse arguments in main for Authorize
2016-01-07 20:31:23 +00:00
bcbd30bb8a
Add easier headless configuration.
...
This will allow setting up a remote with copy&paste of values to a headless machine. It will allow copy+pasting a token into the configuration.
This requires rclone to be on a machine with a proper browser. Custom client id and secrets are supported.
To test token generation, use `rclone auth "fs type"`.
2016-01-07 20:31:23 +00:00
4ce2a84df0
Document workaround for ACD maximum file size.
...
Document workaround for ACD maximum file size and display a warning in verbose mode before upload starts.
Fixes #215 .
2016-01-05 17:12:16 +00:00
b257de4aba
Be more constistent with naming in remotes
...
* External objects are called Fs and Object
* Object.fs always points to the Fs
2015-11-07 11:14:46 +00:00
2fcb8f5db7
Add support for Microsoft One Drive - fixes #10
...
* Still to do
* Copy
* Move
* MoveDir
2015-11-07 10:19:10 +00:00
ff16e0f6df
Factor common error handling into fs module
2015-11-06 15:26:58 +00:00
bf562d7373
Tweak wording on client/secret ids in acd, gcs and drive - fixes #155
2015-10-03 16:04:45 +01:00
e9c915e6fe
Fix golint warnings
2015-09-22 18:47:16 +01:00
05050d53ad
Implement compliant pacing scheme for Amazon Cloud Drive
...
* Implement switchable pacing algorithm
* Add tests for pacer
2015-09-21 21:49:54 +01:00
bc5b63ffef
Move ACD Connection-limiter into Pacer.
2015-09-17 18:42:10 +01:00
0872ec3204
Allow user to override credentials again in drive, gcs and acd - fixes #139
2015-09-16 20:08:40 +01:00
235cbe0e57
amazon cloud drive: retry 409 errors too
2015-09-14 21:00:44 +01:00
00afe6cc96
ACD: Shorten minimum and maximum sleep.
2015-09-14 18:22:21 +01:00
dd48e62b7e
Limit concurrent directory listings.
...
Never run more than "fs.Config.Checkers" directory listings at once.
2015-09-14 18:11:59 +01:00
a1a780e847
Read folders in separate goroutines.
...
As proposed in the FIXME, read folders in parallel.
This appears to fix "Next token is expired" on very big directories.
The only downside is that this doesn't abort at once if an error is found.
I added some logging, so there is some output for "-v".
2015-09-14 18:11:59 +01:00
fa87077211
Retry if a timeout occurs.
...
This happens more rarely than 500, but can still be encountered.
2015-09-14 18:02:35 +01:00
6ac7145d2d
Retry when we get a "500 Internal server error".
...
It seems like this happens randomly, and retrying works fine for that.
2015-09-14 18:02:35 +01:00
f1226f19b2
drive, googlecloudstorage: optionally use auto config for the oauth token
2015-09-12 14:17:39 +01:00
073d112204
Factor pacer module from Drive and use it in Amazon Cloud Drive for
...
smooth API pacing and retry logic.
2015-09-11 19:18:41 +01:00
14f814b806
acd: Retry on 429 errors with backoff and fix upload of 0 length files
2015-09-09 23:23:37 +01:00
8c3df224ef
Implement Amazon Cloud Drive - fixes #45
...
* Optional interfaces Copier, Mover, DirMover not done
2015-09-08 21:02:48 +01:00
