PERF: Avoid using ObjectSpace.each_object in Jobs::Onceoff.enqueue_all (#28072)

We are investigating a memory leak in Sidekiq and saw the following line
when comparing heap dumps over time.

`Allocated IMEMO 14775 objects of size 591000/7389528 (in bytes) at:
/var/www/discourse/app/jobs/onceoff/onceoff.rb:36`

That line in question was doing a `.select { |klass| klass < self  }` on
`ObjectSpace.each_object(Class)`. This for some reason is allocating a
whole bunch of `IMEMO` objects which are instruction sequence objects.

Instead of diving deeper into why this might be leaking, we can just
save our time by switching to an implementation that is more efficient
and does not require looping through a ton of objects.
This commit is contained in:
Alan Guo Xiang Tan
2024-07-25 13:30:56 +08:00
committed by GitHub
parent 3838514d4e
commit f4d06f195d
2 changed files with 14 additions and 11 deletions

View File

@ -3,6 +3,15 @@
class Jobs::Onceoff < ::Jobs::Base class Jobs::Onceoff < ::Jobs::Base
sidekiq_options retry: false sidekiq_options retry: false
class << self
attr_reader :onceoff_job_klasses
def inherited(klass)
@onceoff_job_klasses ||= Set.new
@onceoff_job_klasses << klass
end
end
def self.name_for(klass) def self.name_for(klass)
klass.name.sub(/\AJobs\:\:/, "") klass.name.sub(/\AJobs\:\:/, "")
end end
@ -31,12 +40,9 @@ class Jobs::Onceoff < ::Jobs::Base
def self.enqueue_all def self.enqueue_all
previously_ran = OnceoffLog.pluck(:job_name).uniq previously_ran = OnceoffLog.pluck(:job_name).uniq
ObjectSpace self.onceoff_job_klasses.each do |klass|
.each_object(Class) job_name = name_for(klass)
.select { |klass| klass < self } Jobs.enqueue(job_name.underscore.to_sym) if previously_ran.exclude?(job_name)
.each do |klass| end
job_name = name_for(klass)
Jobs.enqueue(job_name.underscore.to_sym) if previously_ran.exclude?(job_name)
end
end end
end end

View File

@ -7,9 +7,6 @@ RSpec.describe ::Jobs::Onceoff do
require_relative "../../app/jobs/onceoff/" + File.basename(f) require_relative "../../app/jobs/onceoff/" + File.basename(f)
end end
ObjectSpace described_class.onceoff_job_klasses.each { |job| job.new.execute_onceoff(nil) }
.each_object(Class)
.select { |klass| klass.superclass == ::Jobs::Onceoff }
.each { |job| job.new.execute_onceoff(nil) }
end end
end end