Commit Graph

1956 Commits

Author SHA1 Message Date
0fa700b3cf Make integration tests use go1.7+ nested tests #2154
* Removed generated code and code generator
  * Updated docs on how to write integration tests
  * Tidied up the actual integration tests
2018-04-08 15:04:14 +01:00
42f0963bf9 local: retry remove on Windows sharing violation error #2202
Before this change asynchronous closes in cmount could cause sharing
violations under Windows on Remove which manifest themselves
frequently as test failures.

This change lets the Remove be retried on a sharing violation under
Windows.
2018-04-07 17:36:26 +01:00
be54fd8f70 Remove builds conditional on go1.7 since that is now guaranteed #2154
Old fallback code was deleted and the go1.7 style code inlined where
appropriate.
2018-04-07 11:42:55 +01:00
e5be471ce0 Use io.SeekStart/End/Current constants now for go1.7+ #2154 2018-04-07 11:42:36 +01:00
80588a5a6b Replace "golang.org/x/net/context" with "context" for go1.7+ #2154 2018-04-07 11:42:08 +01:00
c749cf8d99 dropbox: fix repeatedly uploading the same files - fixes #2218
In #2134 and dfd0f4c5a458b88857be4a035d910ffc5934f6eb some testing
changes got committed by accident which caused this regression.

This patch reverts it to how it was before.
2018-04-06 15:34:56 +01:00
0cb5c4aa73 gcs: detect bucket presence by listing it - fixes #2193
Doing it like this enables the use of a service account that only has
the "Storage Object Admin" role.
2018-04-06 12:45:15 +01:00
92c5aa3786 s3: add --s3-chunk-size option - fixes #2203 2018-04-05 15:40:08 +01:00
fbe1c7f1ea dropbox: remove unused code 2018-04-05 15:23:23 +01:00
c4531daa43 local: work on spurious "can't copy - source file is being updated" errors #2206
Update all the time comparisons to use time.Time.Equal instead of ==

Improve the logging for that error so we can see exactly what has changed
2018-04-05 14:57:30 +01:00
6e11a25df5 cache: flush the memory cache after close 2018-04-04 23:25:53 +03:00
1dea99ab20 cache: purge file data on notification 2018-04-03 23:24:45 +03:00
e42cee5e02 cache: always forget parent dir for notifications - for #2117 2018-03-31 12:44:09 +03:00
a8267d1628 link: allow creating public link to files and folders - closes #1562 2018-03-29 09:10:19 +02:00
9df266a6b4 onedrive: Fix socket leak in multipart session upload
This had gone unnoticed until recently when we changed to uploading
all files with a multipart session.
2018-03-28 21:03:19 +01:00
4d553ef701 drive: when initialized with a filepath, optional features used incorrect root path – see #2182 2018-03-28 20:33:39 +02:00
b52e34ef5e cache: add info log on notification - for #2150 2018-03-23 22:41:01 +02:00
0ed0d9a7bc cache: integrate with Plex websocket 2018-03-22 21:21:03 +02:00
d9c13bff83 add rc cache/stats 2018-03-22 21:16:16 +02:00
5ba5be9b37 gcs: ignore zero length directory markers at the root too 2018-03-21 20:10:00 +00:00
e9a2cbec37 s3: ignore zero length directory markers at the root too 2018-03-21 20:09:37 +00:00
f6020f1308 gcs: ignore zero length directory markers 2018-03-19 17:42:27 +00:00
a46f2a9eb7 s3: ignore zero length directory markers - fixes #1621 2018-03-19 17:41:46 +00:00
911a78ce6d sftp: require go1.8+ after github.com/pkg/sftp update 2018-03-19 16:37:40 +00:00
4924ac2f17 cache: reduce log level for plex api - for #2102 2018-03-17 11:57:36 +02:00
d4cca8d9f9 onedrive: fix upload of zero length files #1716
Unfortunately multi part upload can't upload zero length files so
bring back the single part upload for zero length files only.

This was broken when we made all uploads multipart uploads.
2018-03-17 09:48:28 +00:00
645cf5ec0f onedrive: fix wrong upload endpoint and createDate #1716
This fixes the problem introduced by 7f744033d8eae0fa916c3207e05cf56d7708fd09
2018-03-16 19:18:51 +00:00
d1bb8efb88 sftp: follow symlinks correctly - fixes #2145
The sftp library delivers the attributes of the symlink rather than
the object pointed to in directory listings, however when we use Stat
from the library it points to the objects.

Previous to this fix this caused items pointed to by symlinks to be
unusable.

After the fix both symlinked files and directories work as expected.
2018-03-16 15:36:47 +00:00
04e055fc06 sftp: Add --sftp-ask-password flag to prompt for password when needed - #2137 2018-03-15 20:24:30 +00:00
aba43cd3a4 Documention for IBM COS (S3) configuration. 2018-03-15 20:20:43 +00:00
7f744033d8 onedrive: Removed upload cutoff and always do session uploads
Set modtime on copy


Added versioning issue to OneDrive documentation
2018-03-15 20:18:11 +00:00
078d705dbe cache: notify vfs and support crypt in rpc - #2111 2018-03-15 11:39:16 +02:00
5981f9fab5 acd: disable integration tests
We no longer have any working keys for Amazon Cloud Drive so disable
the integration tests.
2018-03-14 22:44:46 +00:00
677971643c cache: add support for rc 2018-03-14 22:58:20 +02:00
d76da1f5fd cache: fix dir cache issue - #2117 2018-03-14 11:08:30 +02:00
89748feaa5 s3: update docs to discourage use of v2 auth - fixes #2120
From testing it appears that CEPH no longer works properly with v2
auth and neither does Dreamhost, so update the docs anc configuration
to recommend v4 auth.
2018-03-13 20:47:29 +00:00
dfd0f4c5a4 sync: when using --backup-dir don't delete files if we can't set their modtime
This is a problem when syncing a file which just needed its modtime
set with dropbox which can't set the mod time of a file without
re-uploading it.

Before this change we would delete the file, then the server side move
would fail moving the file to the backup-dir because it no longer
existed.

After this change the destination file is moved to the backup-dir
instead of being deleted and the new file is uploaded.

Fixes #2134
2018-03-13 16:05:06 +00:00
a7dbf32c53 cache: Implement --cache-db-wait-time flag
This can be used to make the cache wait for other running cache
backends to finish rather than erroring after 1 second.
2018-03-08 21:14:09 +00:00
6025bb6ad1 local: fix race conditions updating the hashes
This was causing occasional test failures for the -race test of mount
and cmount.
2018-03-08 21:08:41 +00:00
70f07fd3ac fs: add ChangeNotify and backend support for it (#2094)
* fs: rename DirChangeNotify to ChangeNotify

* cache: switch to ChangeNotify

* ChangeNotify: keep order of notifications
2018-03-08 22:03:34 +02:00
fe8eeec5b5 cache: improve efficiency with RangeOption and RangeSeek #1825
* All remotes now support RangeOption so remove SeekOption
  * Correct off by one error as RangeOption arguments are inclusive.
  * Use RangeSeek in preference to Seek if available
2018-03-02 17:10:56 +00:00
7d4da1c66a local: fix crash on Stat error while reading a file 2018-03-01 13:17:40 +00:00
f3e982d3bf azureblob,b2,gcs,qingstor,s3,swift: Don't check for bucket/container presense if listing was OK
In a typical rclone copy to a bucket/container based remote, before
this change we were doing a list, followed by a HEAD of the bucket to
check it existed before doing the copy.  The fact the list succeeded
means the bucket exists so mark it OK at that point.

Issue #1421
2018-03-01 12:11:34 +00:00
fdebf9da31 local: Downgrade "invalid cross-device link: trying copy" to debug - Fixes #1875 2018-02-28 21:27:34 +00:00
167a4396c7 drive: remove debug binary 2018-02-27 09:59:06 +01:00
724120d2f3 local: make DirMove return fs.ErrorCantDirMove to allow fallback
Before this change `rclone move localdir /mnt/different-fs` would
error.  Now it falls back to moving individual files, which in turn
falls back to copying individual files across the filesystem boundary.
2018-02-26 12:55:05 +00:00
25bbc5d22b drive: make --drive-auth-owner-only look in all directories
Previously it was ignoring directories which weren't owned by the user
which meant it was ignoring files owned by the user in those
directories.
2018-02-26 12:30:59 +00:00
00adf40f9f cryptdecode: use Cipher instead of NewFs (#2087)
* crypt: extract NewCipher out of NewFs
* cryptdecode: make use of crypt.NewCipher

Fixes #2075
2018-02-25 12:57:14 +01:00
aeefa34f62 fstests: add TestInternal (#2085)
TestInternal allows to perform a custom test on the backend using the
optional InternalTester interface.
2018-02-25 10:58:06 +01:00
1383df4f58 b2: add more logging on multipart upload errors to debug #2036 2018-02-21 09:05:59 +00:00