FEATURE: Add a sidekiq job for syncing S3 ACLs (#16449)

Sometimes we need to update a _lot_ of ACLs on S3 (such as when secure media
is enabled), and since it takes ~1s per upload to update the ACL, this is
best spread out over many jobs instead of having to do the whole thing serially.

In future, it will be better to have a job that can be run based on
a column on uploads (e.g. acl_stale) so we can track progress, similar
to how we can set the baked_version to nil to rebake posts.
This commit is contained in:
Martin Brennan
2022-04-12 14:26:42 +10:00
committed by GitHub
parent 131a4674e3
commit 9f2138dc92
3 changed files with 74 additions and 4 deletions

View File

@ -279,13 +279,24 @@ module FileStore
end
end
def update_upload_ACL(upload)
def update_upload_ACL(upload, optimized_images_preloaded: false)
key = get_upload_key(upload)
update_ACL(key, upload.secure?)
upload.optimized_images.find_each do |optimized_image|
optimized_image_key = get_path_for_optimized_image(optimized_image)
update_ACL(optimized_image_key, upload.secure?)
# if we do find_each when the images have already been preloaded with
# includes(:optimized_images), then the optimized_images are fetched
# from the database again, negating the preloading if this operation
# is done on a large amount of uploads at once (see Jobs::SyncAclsForUploads)
if optimized_images_preloaded
upload.optimized_images.each do |optimized_image|
optimized_image_key = get_path_for_optimized_image(optimized_image)
update_ACL(optimized_image_key, upload.secure?)
end
else
upload.optimized_images.find_each do |optimized_image|
optimized_image_key = get_path_for_optimized_image(optimized_image)
update_ACL(optimized_image_key, upload.secure?)
end
end
true