53ff3b3b32
s3: add Selectel as a provider
2024-10-22 19:54:33 +01:00
589458d1fe
docs: fix Scaleway Glacier website URL
Docker beta build / Build image job (push) Has been cancelled
2024-10-12 12:13:47 +01:00
9614fc60f2
s3: add Outscale provider
...
Docker beta build / Build image job (push) Has been cancelled
Signed-off-by: matthias.gatto <matthias.gatto@outscale.com >
Co-authored-by: André Tran <andre.tran@outscale.com >
2024-10-02 10:26:41 +01:00
192524c004
s3: add initial --s3-directory-bucket to support AWS Directory Buckets
...
This will ensure no Content-Md5 headers are sent and ensure ETags are not
interpreted as MD5 sums. X-Amz-Meta-Md5chksum will be set on all objects
whether single or multipart uploaded.
This also sets "no_check_bucket = true".
This is enough to make the integration tests pass, but there are some
limitations as noted in the docs.
See: https://forum.rclone.org/t/support-s3-directory-bucket/47653/
2024-09-19 12:01:24 +01:00
da70088b11
docs: add section for improving performance for s3
2024-09-12 11:29:35 +01:00
49f69196c2
Version v1.68.0
2024-09-08 16:21:56 +01:00
da25305ba0
docs: update backend config samples
2024-08-17 16:00:18 +02:00
8aef1de695
s3: fix Cloudflare R2 integration tests after SDKv2 update #4989
...
Cloudflare will normally automatically decompress files with
`Content-Encoding: gzip` when downloaded. This is not what AWS S3 does
and it breaks the integration tests.
This fudges the integration tests to upload the test file with
`Cache-Control: no-transform` on Cloudflare R2 and puts a note in the
docs about this problem.
2024-08-07 12:13:13 +01:00
fd9049c83d
s3: update to using AWS SDK v2 - fixes #4989
...
SDK v2 conversion
Changes
- `--s3-sts-endpoint` is no longer supported
- `--s3-use-unsigned-payload` to control use of trailer checksums (needed for non AWS)
2024-08-07 12:13:13 +01:00
9866d1c636
docs: s3: add section on using too much memory #7974
2024-08-01 16:33:09 +01:00
1c9da8c96a
docs: recommend no_check_bucket = true for Alibaba - fixes #7889
...
Change-Id: Ib6246e416ce67dddc3cb69350de69129a8826ce3
2024-06-15 17:39:05 +01:00
93e8a976ef
Version v1.67.0
2024-06-14 16:04:51 +01:00
4568feb5f9
s3: Add Magalu S3 Object Storage as provider
2024-06-06 11:25:45 +01:00
1e3ab7acfd
docs: fix MANUAL formatting problems
...
1) Missing closing code backticks (```) in s3.md causes formatting problems
2) Pandoc requires blank lines before ATX headings
2024-04-05 10:41:47 +02:00
08bf5228a7
docs: Add R2 note about no_check_bucket
2024-04-02 12:08:56 +01:00
f9429de807
s3: update Scaleway's configuration options - fixes #7507
...
In order to handle special character, the configuration must specify
rclone configuration to use `list_url_encode`.
2024-03-31 17:42:20 +01:00
b6013a5e68
Version v1.66.0
2024-03-10 11:22:43 +00:00
0e746f25a3
amazonclouddrive: remove Amazon Drive backend code and docs #7539
...
The Amazon Drive backend is closed from 2023-12-31.
See: https://www.amazon.com/b?ie=UTF8&node=23943055011
2024-01-04 17:05:54 +00:00
c3117d9efb
Doc change: Add the CreateBucket permission requirement for AWS S3
2023-12-07 16:46:04 +00:00
82b963e372
Version v1.65.0
2023-11-26 16:07:39 +00:00
ba11040d6b
s3: detect looping when using gcs and versions
...
Apparently gcs doesn't return an S3 compatible result when using
versions.
In particular it doesn't return a NextKeyMarker - this means rclone
loops and fetches the same page over and over again.
This patch detects the problem and stops the infinite retries but it
doesn't fix the underlying problem.
See: https://forum.rclone.org/t/list-s3-versions-files-looping-bug/42974
See: https://issuetracker.google.com/u/0/issues/312292516
2023-11-23 09:50:28 +00:00
a7faf05393
docs: cleanup backend hashes sections
2023-11-20 17:43:57 +00:00
93f35c915a
serve s3: pre-merge tweaks
...
- Changes
- Rename `--s3-authkey` to `--auth-key` to get it out of the s3 backend namespace
- Enable `Content-MD5` integrity checks
- Remove locking after code audit
- Documentation
- Factor out documentation into seperate file
- Add Quickstart to docs
- Add Bugs section to docs
- Add experimental tag to docs
- Add rclone provider to s3 backend docs
- Fixes
- Correct quirks in s3 backend
- Change fmt.Printlns into fs.Logs
- Make metadata storage per backend not global
- Log on startup if anonymous access is enabled
- Coding style fixes
- rename fs to vfs to save confusion with the rest of rclone code
- rename db to b for *s3Backend
Fixes #7062
2023-11-16 16:59:56 +00:00
eb3082a1eb
s3: add Linode provider
2023-09-23 14:34:00 +01:00
77f7bb08af
Version v1.64.0
2023-09-11 15:59:44 +01:00
363da9aa82
docs: s3 versions names caveat
2023-07-27 12:36:50 +09:00
33fec9c835
doc: Fix Leviia block
2023-07-18 19:58:19 +01:00
119ccb2b95
s3: add Leviia S3 Object Storage as provider
2023-07-16 18:08:47 +01:00
f1a8420814
s3: add synology to s3 provider list
2023-07-06 10:54:07 +01:00
e90537b2e9
Version v1.63.0
2023-06-30 14:11:17 +01:00
8274712c2c
docs: s3: fix example for restoring single objects
...
See: https://forum.rclone.org/t/cant-restore-files-from-aws-glacier-deep-only-directories/39258/3
2023-06-26 11:41:15 +01:00
666e34cf69
s3: docs: old broken link updated
2023-06-09 18:15:54 +01:00
f226f2dfb1
s3: add petabox.io to s3 providers
2023-05-05 09:44:25 +01:00
0487e465ee
docs: s3: clarify that X-Amz-Meta-Md5chksum is really a base64-encoded hex
2023-04-25 11:39:36 +01:00
54a9488e59
s3: add GCS to provider list
2023-03-16 14:24:21 +00:00
77c7077458
Version v1.62.0
2023-03-14 12:42:23 +00:00
ffd4ab222c
docs: add idrive e2 as a major sponsor
2023-03-14 12:37:34 +00:00
b9d9f9edb0
docs: use --interactive instead of -i in examples to avoid confusion
2023-01-24 20:43:51 +01:00
2001cc0831
Version v1.61.0
2022-12-20 17:16:14 +00:00
c6ad15e3b8
s3: make DigitalOcean name canonical
2022-12-14 16:35:05 +00:00
0a8b1fe5de
s3: add Liara LOS to provider list
2022-12-06 12:25:23 +00:00
959cd938bc
docs: Add minimum versions to all the backend pages and some of the other pages
2022-11-18 14:41:24 +00:00
01dbbff62e
Version v1.60.0
2022-10-21 15:06:08 +01:00
66ed0ca726
s3: add Qiniu KODO to s3 provider list - fixes #6195
2022-10-13 15:49:22 +01:00
c080b39e47
s3: add support for IONOS Cloud Storage
2022-09-15 16:04:34 +01:00
ce3b65e6dc
all: fix spelling across the project
...
* abcdefghijklmnopqrstuvwxyz
* accounting
* additional
* allowed
* almost
* already
* appropriately
* arise
* bandwidth
* behave
* bidirectional
* brackets
* cached
* characters
* cloud
* committing
* concatenating
* configured
* constructs
* current
* cutoff
* deferred
* different
* directory
* disposition
* dropbox
* either way
* error
* excess
* experiments
* explicitly
* externally
* files
* github
* gzipped
* hierarchies
* huffman
* hyphen
* implicitly
* independent
* insensitive
* integrity
* libraries
* literally
* metadata
* mimics
* missing
* modification
* multipart
* multiple
* nightmare
* nonexistent
* number
* obscure
* ourselves
* overridden
* potatoes
* preexisting
* priority
* received
* remote
* replacement
* represents
* reproducibility
* response
* satisfies
* sensitive
* separately
* separator
* specifying
* string
* successful
* synchronization
* syncing
* šenfeld
* take
* temporarily
* testcontents
* that
* the
* themselves
* throttling
* timeout
* transaction
* transferred
* unnecessary
* using
* webbrowser
* which
* with
* workspace
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com >
2022-08-30 11:16:26 +02:00
4344a3e2ea
s3: implement --s3-version-at flag - Fixes #1776
2022-08-05 16:45:23 +01:00
81d242473a
s3: implement Purge to purge versions and backend cleanup-hidden
2022-08-05 16:45:23 +01:00
0ae171416f
s3: implement --s3-versions flag - See #1776
2022-08-05 16:45:23 +01:00
7243918069
s3: implement backend versioning command to get/set bucket versioning
2022-08-05 16:42:30 +01:00
