Commit Graph

142 Commits

Author SHA1 Message Date
5b934cb33d FIX: Error when trying to move the same file to tombstone.
If an optimized image is destroyed when a previous similar optimized
image is already placed in the tombstone, `FileUtils.move` will blow up.
2019-04-24 16:47:36 +08:00
243fb8d9ad Fix the build. 2019-03-13 17:39:07 +08:00
b0c8fdd7da FIX: Properly support defaults for upload site settings. 2019-03-13 16:36:57 +08:00
664e90bd17 FIX: ensure local images use local CDN when uploads are stored on S3
When the S3 store was enabled, we were only applying the S3 CDN.
So all images stored locally, like the emojis, were never put on the local CDN.

Fixed a bunch of CookedPostProcessor test by adding a call to 'optimize_urls'
in order to get final URLs.

I also removed the unnecessary PrettyText.add_s3_cdn method since this is already
handled in the CookedPostProcessor.
2019-02-20 19:24:38 +01:00
563b953224 DEV: Add 'backfill_etags_' to the method name since it also backfilling the etags 2019-02-19 21:54:35 +05:30
0472bd4adc FIX: Remove 'backfill_etags' keyword argument from 'uploads:missing' rake task
And etags backfilling code is optimized
2019-02-15 00:34:35 +05:30
7b5931013a Update rake task to backfill etags from s3 inventory 2019-02-14 05:18:06 +05:30
b4f713ca52 FEATURE: Use amazon s3 inventory to manage upload stats (#6867) 2019-02-01 10:10:48 +05:30
f94c0283b2 FIX: Use correct version when generating file path for optimized image (#6871) 2019-01-11 18:35:38 +05:30
75dbb98cca FEATURE: Add S3 etag value to uploads table (#6795) 2019-01-04 14:16:22 +08:00
5381096bfd PERF: new 'migrate_to_s3' rake task 2018-12-26 17:34:49 +01:00
cae5ba7356 FIX: Ensure that multisite s3 uploads are tombstoned correctly (#6769)
* FIX: Ensure that multisite uploads are tombstoned into the correct paths

* Move multisite specs to spec/multisite/s3_store_spec.rb
2018-12-19 13:32:32 +08:00
503ae1829f FIX: All multisite upload paths should start with /uploads/default/.. (#6707) 2018-12-03 12:04:14 +08:00
871d4543cc FIX: Use File.join for relative_base_url, fix spec 2018-11-29 09:49:56 +05:30
05a4f3fb51 FEATURE: Multisite support for S3 image stores (#6689)
* FEATURE: Multisite support for S3 image stores

* Use File.join to concatenate all paths & fix linting on multisite/s3_store_spec.rb
2018-11-29 12:11:48 +08:00
bcdf5b2f47 DEV: improve missing uploads query and skip checking file size 2018-11-27 02:21:33 +05:30
4ccf9d28eb Remove trailing whitespaces 2018-11-27 01:15:29 +05:30
fd272eee44 FEATURE: Make uploads:missing task compatible with s3 uploads 2018-11-27 00:54:51 +05:30
ce6a0a5e9e FIX: Moving upload to tombstone should update modification time.
A upload created a long time ago will be nuked from the tombstone
immediately if it gets deleted.
2018-09-18 10:48:29 +08:00
e1b16e445e Rename FileHelper.is_image? -> FileHelper.is_supported_image?. 2018-09-12 09:22:28 +08:00
8496537590 Add RECOVER_FROM_S3 to uploads:list_posts_with_broken_images rake task. 2018-09-10 15:14:30 +08:00
Sam
5d96809abd FIX: improve support for subfolder S3 CDN 2018-08-22 12:31:13 +10:00
Sam
f5142861e5 Revert "Revert "FIX: upload URLs from S3 on subfolder installs""
This reverts commit 26c96e97e5e30b1ca4e7b73103ca02c908ed4073.

We have no choice but to run this code
2018-08-22 11:31:33 +10:00
Sam
26c96e97e5 Revert "FIX: upload URLs from S3 on subfolder installs"
This reverts commit 357df2ff4fa1afc3c1efa4d33288aedd24882d88.
2018-08-22 10:51:40 +10:00
357df2ff4f FIX: upload URLs from S3 on subfolder installs 2018-08-21 14:58:55 -04:00
aafff740d2 Add FileStore::S3Store#copy_file. 2018-08-08 11:30:34 +08:00
dba22bbde2 rollback changes
This reverts:
* 1baba84c438e "fix s3 subfolders harder"
* ea5e57938edf "fix test for absolute_base_url change"
2018-07-06 17:16:40 -05:00
52e9f49ec1 fix s3 subfolders harder
specifically, include the folder in absolute_base_url
2018-07-06 16:28:40 -05:00
448e2fe1a2 FIX: properly delete files in the download cache 2018-07-04 18:18:39 +02:00
4be0e31459 fix s3_cdn_url when the s3 bucket contains a folder 2018-05-23 15:51:02 -05:00
5f4f617689 FIX: cache_file storage cleanup logic was wrong
https://meta.discourse.org/t/68296
2018-01-18 17:00:04 +01:00
Sam
70bb2aa426 FEATURE: allow specifying s3 config via globals
This refactors handling of s3 so it can be specified via GlobalSetting

This means that in a multisite environment you can configure s3 uploads
without actual sites knowing credentials in s3

It is a critical setting for situations where assets are mirrored to s3.
2017-10-06 16:20:01 +11:00
8cc8010564 Maintain backwards compatibility before Jobs::MigrateUploadExtensions runs. 2017-08-03 11:56:55 +09:00
83011045c8 fix rubocop offenses 2017-07-31 11:59:16 -04:00
5d528f0d15 Merge pull request #4958 from dmacjam/search_posts_by_filetype
FEATURE: Search posts by filetype
2017-07-31 11:55:34 -04:00
5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
d8c27e3871 Merge branch 'master' into search_posts_by_filetype 2017-07-25 14:41:20 -04:00
677267ae78 Add onceoff job for uploads migration of column extension. Simplify filetype search and related rspec tests. 2017-07-12 17:19:27 +02:00
8c445e9f17 Fix backend code for searching by a filetype as a combination of uploads and topic links. Add rspec test for extracting file extension in upload. 2017-07-06 19:19:31 +02:00
5d63a7f4a6 FIX: pull hotlinked images even when they have no extension 2017-06-13 13:27:05 +02:00
cdbe027c1c Refactor FileHelper to use keyword arguments. 2017-05-24 13:54:26 -04:00
f534f041a0 FIX: Ensure directory exists. 2017-04-07 15:50:17 +08:00
da7a44064b Fix purge_tombstone for the brave new world of secure command execution 2017-03-22 10:27:07 +11:00
e7c972ac89 FIX: Don't use backticks that take in inputs. 2017-03-17 15:33:51 +08:00
93dfc87b99 FIX: always set the 'content_type' when storing a file on S3 2016-10-17 19:16:29 +02:00
3141c179f7 REFACTOR: Get bucket name from S3Helper. 2016-08-19 14:08:37 +08:00
7ff1f6cb9d Allow custom bucket name for FileStore::S3Store. 2016-08-16 15:25:42 +08:00
205be0d044 Remove unused require. 2016-08-15 21:58:55 +08:00
0433163866 FEATURE: Support subfolders in SiteSetting.s3_backup_bucket. 2016-08-15 16:14:51 +08:00
aa5de3c40a FEATURE: Support subfolders in S3 bucket name.
This commit also fixes a bug where s3 uploads are not
moved to a tombstone folder when removed.
2016-08-15 13:07:41 +08:00