FIX: Unicorn master and Sidekiq reopening logs at the same time (#29137)

In our production environment, we have been seeing Sidekiq processes
getting stuck randomly when a USR1 signal is sent to the Unicorn master
process. We have not been able to identify the root cause of why the
Sidekiq process gets stuck. We however noticed that when the Unicorn
master process receives a USR1 signal, it will reopen the logs for the
Unicorn master process first before sending a USR1 signal for the
Unicorn worker processes to reopen the logs. We figured that we should
do the same for the Sidekiq process as well when a USR1 signal.

In this commit, we introduce an arbitrary delay of 1 second before we
the Sidekiq process reopens its log files so as to allow enough time for the Unicorn
master to finish reopening it logs first.

We also do not send reopen logs for the Sidekiq process if the `DISCOURSE_LOG_SIDEKIQ`
env is not present because there is no need to reopen any logs.
This commit is contained in:
Alan Guo Xiang Tan
2024-10-10 08:01:40 +08:00
committed by GitHub
parent 08d5cf01cd
commit c1f25cdf5b
4 changed files with 42 additions and 20 deletions

View File

@ -1223,4 +1223,8 @@ module Discourse
) if SiteSetting.set_locale_from_accept_language_header
locale
end
def self.enable_sidekiq_logging?
ENV["DISCOURSE_LOG_SIDEKIQ"] == "1"
end
end