github-mirror/discourse
2
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2025-05-23 14:01:14 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

DEV: Introduce hidden s3_inventory_bucket site setting (#27304)

Browse Source 
This commit introduces a hidden `s3_inventory_bucket` site setting which
replaces the `enable_s3_inventory` and `s3_configure_inventory_policy`
site setting.

The reason `enable_s3_inventory` and `s3_configure_inventory_policy`
site settings are removed is because this feature has technically been
broken since it was introduced. When the `enable_s3_inventory` feature
is turned on, the app will because configure a daily inventory policy for the
`s3_upload_bucket` bucket and store the inventories under a prefix in
the bucket. The problem here is that once the inventories are created,
there is nothing cleaning up all these inventories so whoever that has
enabled this feature would have been paying the cost of storing a whole
bunch of inventory files which are never used. Given that we have not
received any complains about inventory files inflating S3 storage costs,
we think that it is very likely that this feature is no longer being
used and we are looking to drop support for this feature in the not too
distance future.

For now, we will still support a hidden `s3_inventory_bucket` site
setting which site administrators can configure via the
`DISCOURSE_S3_INVENTORY_BUCKET` env.
This commit is contained in:
Alan Guo Xiang Tan
2024-06-10 13:16:00 +08:00
committed by GitHub
parent 3bff633ce0
commit 8cf4ed5f88
13 changed files with 62 additions and 260 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
spec/lib/s3_inventory_multisite_spec.rb
View File

@ -5,15 +5,16 @@ require "s3_inventory"
require "file_store/s3_store"
RSpec.describe "S3Inventory", type: :multisite do
let(:client) { Aws::S3::Client.new(stub_responses: true) }
let(:helper) { S3Helper.new(SiteSetting.Upload.s3_upload_bucket.downcase, "", client: client) }
let(:inventory) { S3Inventory.new(helper, :upload) }
let(:inventory) do
S3Inventory.new(:upload, s3_inventory_bucket: "some-inventory-bucket/some/prefix")
end
let(:csv_filename) { "#{Rails.root}/spec/fixtures/csv/s3_inventory.csv" }
it "can create per-site files" do
freeze_time
setup_s3
SiteSetting.enable_s3_inventory = true
inventory.stubs(:files).returns([{ key: "Key", filename: "#{csv_filename}.gz" }])
inventory.stubs(:cleanup!)
@ -23,6 +24,7 @@ RSpec.describe "S3Inventory", type: :multisite do
expect(db1.lines.count).to eq(3)
expect(db2.lines.count).to eq(1)
files.values.each do |f|
f.close
f.unlink