Before this change, we were relying on the `lograge`'s gem default
`log_level` of `info`. However, `lograge` should be following
`Rails.application.config.log_level` instead and this commits seeks to
correct that behaviour.
Introduction of system themes. System themes are local themes which:
- Cannot be deleted;
- Cannot have “custom code” added, components only;
- Cannot have uploads;
- Cannot edit color palettes;
- Are updated on deploy, like core plugins.
This PR added the Foundation system theme, which is an empty theme like
Default. The Foundation theme will be added in the next PR.
In a development environment, when system theme files are
changed/added/deleted, the theme is reuploaded and the page is reloaded
to make it a good experience for the engineer working on improvements.
System themes are not visible until
`SiteSetting.experimental_system_theme` is enabled.
In production, the enforce_hostname middleware overwrites the HTTP_HOST
env using `Discourse.base_url_no_prefix`, which takes into account any
configured protocol/hostname/post overrides.
That middleware is not used in development, so if we want omniauth to
respect any host/port overrides, we need to configure the 'full host'
directly.
Discourse message-bus traffic is not considered a 'public api' for
general consumption. However, it does make sense to have consistency
with the CORS behavior of the rest of the app, so that people can use it
at their own risk.
Currently, if you configure a webhook with reviewable events and apply
categories/tags filtering, no webhook gets fired for reviewable events.
This is because when we schedule the `EmitWebHookEvent` job, we don't
pass to it the reviewable's category or tags, making it seem like the
reviewable doesn't belong to the filtering category/tags that webhook
specifies.
This commit ensures rails will recognise `IST` as a timezone. It will be
mapped to the standard timezone `Asia/Kolkata`.
Its technically not a standard, but it's used by many people so we are
adding it as a timezone in core.
/t/-/150799
This improves the error message(s) displayed when an error happens while
using a social login to log in / sign up into a Discourse community.
Unfortunately, we can't be super precise in the reason behind the error
(it can be the user clicked "cancel" during the authorization phase, or
any of the plethora of other possible errors) because the reason isn't
provided (either for security reasons, or because it's just hard to do
so in a reliable & consistent way accross all social logins).
The best I could do was to
- add the name of the "social login" provider in the error message
- tweak the copy a bit for the different cases handle
- fix the CSS/HTML to match the one used by the main application
In order to show the name of the provider, we use either the "provider"
or the "strategy" query parameter, or use the name of the authenticator
(if it's the only one enabled).
Internal ref - t/153662
---
**BEFORE**
**AFTER**
Ignore any files/directories in '.devenv' like we do for those in
'node_modules' to 1) speed-up watching files and 2) prevent files from
being watched multiples times (due to symlinks).
No specs since this is only a warning that is being shown in development
when using devenv.
---
Here's an example of the 100s of warning you get when using devenv
```plain
** ERROR: directory is already being watched! **
Directory: /Users/zogstrip/Poetry/discourse/plugins/discourse-login-client/.devenv/profile
is already being watched through: /nix/store/jp4qhxc2f98nyq90kia200mrygl2ndcy-devenv-profile
MORE INFO: https://github.com/guard/listen/blob/master/README.md
```
When Translation Overrides are changed, a message is sent on the
`/i18n-flush` channel to ensure all running unicorns flush their caches.
This message is not received in a database-specific context, so running
`ExtraLocalesController.clear_cache!` was only affecting the default
site in a multisite cluster. This commit updates it to flush caches for
all sites.
We are no longer using any of the transpilation/bundling features of
Sprockets. We only use it to serve assets in development, and then
collect & fingerprint them in production. This commit switches us to use
the more modern "Propshaft" gem for that functionality.
Propshaft is much simpler than Sprockets. Instead of taking a
combination of paths + "precompile" list, Propshaft simply assumes all
files in the configured directory are required in production. Previously
we had some base paths configured quite high in the directory structure,
and then only precompiled selected assets within the directory. That's
no longer possible, so this commit refactors those places (mostly
plugin-related) to use dedicated directories under
`app/assets/generated/`.
Another difference is that Propshaft applies asset digests in
development as well as production. This is great for caching & dev/prod
consistency, but does mean some small changes were required in tests.
We previously had some freedom-patches applied to Sprockets. Some of
those had to be ported across to Propshaft. We now have three patches:
1. Skip adding digest hashes to webpack-generated chunks (which are
already digested, and referred to from other js files)
2. Avoid raising errors for missing assets in test mode. We don't always
compile assets before running basic RSpec tests.
3. Maintain relative paths for sourcemap URLs, so that files don't need
to be recompiled depending on their CDN path
Significant refactors are made to the `assets.rake` and `s3.rake` tasks,
which rely on implementation details of Sprockets/Propshaft.
This JS was used to wrap long usernames. Nowadays in core, we use a
simpler `word-break: break-all;` pattern, so it makes sense to use the
same simplification in the embedded comments view.
Previously all locale bundles would be built & compressed during
assets:precompile. For most sites, only one of these languages was
actually used, so this is fairly wasteful.
This commit moves the main locale bundle into the
ExtraLocalesController, which has recently undergone many improvements
to make it more efficient. This allows locale files to be bundled "just
in time" when they're first accessed.
Now that brotli level=6 is enabled for these assets in our nginx config,
this change should have no impact on the locale bundle size.
This commit adds a `:sidekiq_job_error` event which is
triggered when an error is encountered while running a Sidekiq job. The
intent of this change is to complement the `:sidekq_job_ran` event which
can be used with the `:sidekiq_job_error` event to calculate the error
rate for each job.
In a development environment, the server auto-restarter can't watch symlinked paths, which means that any symlinked plugins won't reload if their non-autoloaded files change.
This change does a simple resolve of the symlink for any symlinked plugins, and adds the real plugin path to those that it will auto-restart for.
The service worker isn't served via normal asset paths or the CDN.
Instead, the ERB was being compiled by sprockets, fished out of the
`public/` directory by the static_controller, and then the
sprockets-specific stuff like `sourceMappingUrl` was being removed.
Instead, we can put the ERB under `views/static/`, and have it evaluate
at runtime. There are only a couple of super-cheap interpolations, plus
the route is cached in nginx, so there is no performance concern.
This takes us one step closer to removing sprockets.
As we’re currently using a namespace for Sidekiq, in order to upgrade to
the latest version, we need to drop it as it’s not supported anymore.
The recommended way is to use a different Redis DB for Sidekiq.
This patch uses a different config for Sidekiq and also takes care of
migrating existing jobs (in queues and the retry and scheduled sets).
This change standardises the `User-Agent` header that Discourse will send when talking to other sites.
`Discourse.user_agent` is now the authority on what the user agent value should be. For Onebox requests, this changes the user agent from their existing value to match the new value (unless overridden).
For all other requests, `Net::HTTPHeader` is monkey-patched to add a default `User-Agent` header when one hasn't been provided.
This change expands the matcher added in 02bebb7e9152996d86b980baea73eacb5d9222ef to include plugin `settings.yml` files as requiring a server restart when they change.
The `Listen.to(only: ...)` filter only matches against filenames (as opposed to full paths) so we can't add a matcher against `plugins/.*/config/settings.yml` to make it explicit.
site_settings.yml is read when the server starts, but isn't re-read if it changes. (Eg, adding a new setting, or changing the configuration of an existing setting.)
This change includes site_settings.yml as a file that requires a server restart when it changes. This behaviour obeys the AUTO_RESTART environment variable.
This commit adds the `add_request_rate_limiter` plugin API which allows plugins to add custom rate limiters on top of the default rate limiters which requests by a user's id or the request's IP address.
Example to add a rate limiter that rate limits all requests from Googlebot under the same rate limit bucket:
```
add_request_rate_limiter(
identifier: :country,
key: ->(request) { "country/#{DiscourseIpInfo.get(request.ip)[:country]}" },
activate_when: ->(request) { DiscourseIpInfo.get(request.ip)[:country].present? },
)
```
The Listen gem watches recursively, which has a cost per-file on Linux (via rb-inotify). This commit skips a bunch of unnecessary directories to reduce the startup cost.
### Why?
Before, all flags were static. Therefore, they were stored in class variables and serialized by SiteSerializer. Recently, we added an option for admins to add their own flags or disable existing flags. Therefore, the class variable had to be dropped because it was unsafe for a multisite environment. However, it started causing performance problems.
### Solution
When a new Flag system is used, instead of using PostActionType, we can serialize Flags and use fragment cache for performance reasons.
At the same time, we are still supporting deprecated `replace_flags` API call. When it is used, we fall back to the old solution and the admin cannot add custom flags. In a couple of months, we will be able to drop that API function and clean that code properly. However, because it may still be used, redis cache was introduced to improve performance.
To test backward compatibility you can add this code to any plugin
```ruby
replace_flags do |flag_settings|
flag_settings.add(
4,
:inappropriate,
topic_type: true,
notify_type: true,
auto_action_type: true,
)
flag_settings.add(1001, :trolling, topic_type: true, notify_type: true, auto_action_type: true)
end
```
### Why?
Before, all flags were static. Therefore, they were stored in class variables and serialized by SiteSerializer. Recently, we added an option for admins to add their own flags or disable existing flags. Therefore, the class variable had to be dropped because it was unsafe for a multisite environment. However, it started causing performance problems.
### Solution
When a new Flag system is used, instead of using PostActionType, we can serialize Flags and use fragment cache for performance reasons.
At the same time, we are still supporting deprecated `replace_flags` API call. When it is used, we fall back to the old solution and the admin cannot add custom flags. In a couple of months, we will be able to drop that API function and clean that code properly. However, because it may still be used, redis cache was introduced to improve performance.
To test backward compatibility you can add this code to any plugin
```ruby
replace_flags do |flag_settings|
flag_settings.add(
4,
:inappropriate,
topic_type: true,
notify_type: true,
auto_action_type: true,
)
flag_settings.add(1001, :trolling, topic_type: true, notify_type: true, auto_action_type: true)
end
```
This commit adds a `MiniSchedulerLongRunningJobLogger` class which will
poll every 60 seconds for mini_scheduler jobs which are stuck. When it
detects that a job is stuck, it will log a warning message with the
current backtrace of the thread that is executing the job.
Note that for scheduled jobs which are executed at a frequency of less
than 30 minutes, we will log when the job has been executing for 30
minutes.
For scheduled jobs executed at a frequency of less than 2 hours, we will
log when the job has been executing for a duration greater than its
specified frequency.
For scheduled jobs executed at a frequency greater than 2 hours, we will
log as long as the job has been executing for more than 2 hours.
This commit patches `Net::HTTP` to reduce the default timeouts of 60
seconds when we are processing a request. There are certain routes in
Discourse which makes external requests and if the proper timeouts are
not set, we risk having the Unicorn master process force restarting the
Unicorn workers once the `30` seconds timeout is reached. This can
potentially become a vector for DoS attacks and this commit is aimed at
reducing the risk here.
Previously, we couldn't change the user agent name dynamically for onebox requests. In this commit, a new hidden site setting `onebox_user_agent` is created to override the default user agent value specified in the [initializer](c333e9d6e6/config/initializers/100-onebox_options.rb (L15)).
Co-authored-by: Régis Hanol <regis@hanol.fr>
`after_commit` should be used before refreshing processes to be sure that the database is already updated.
Also, MessageBus is used instead of events as MessageBus works correctly with many processes;
This site setting has always been experimental and hidden since it was
added 7 years ago. Drop it to simplify the way we enable logging in a
logstash friendly way.
This commit updates the `101-lograge` initializer to unsubscribe from
log events logged by `ActionView::LogSubscriber`. This is what the `lograge`
gem has been doing but its implementation is not compatible with Rails
7.1 and we started getting noise in our logs when lograge is enabled.
This commit monkey patches `Rails::Rack::Logger` to not log reqeust
information like `Started GET "/service-worker.js" for 127.0.0.1 at 2024-07-05 15:28:12 +0800`
when lograge is enabled. This was previously excluded by a monkey patch
in the `lograge` gem but that monkey patch has since broke and the gem
is unmaintained.
This reverts commit 8e10878e1a37a608b0e1705bf9d0a06a295b8a63.
Something is broken on a friday so reverting first before I pick this up
again next Monday.
This commit rewrites `DiscourseLogstashLogger` to not be an instance
of `LogstashLogger`. The reason we don't want it to be an instance of
`LogstashLogger` is because we want the new logger to be chained to
Logster's logger which can then pass down useful information like the
request's env and error backtraces which Logster has already gathered.
Note that this commit does not bother to maintain backwards
compatibility and drops the `LOGSTASH_URI` and `UNICORN_LOGSTASH_URI`
ENV variables which were previously used to configure the destination in
which `logstash-logger` would send the logs to. Instead, we introduce
the `ENABLE_LOGSTASH_LOGGER` ENV variable to replace both ENV and remove
the need for the log paths to be specified. Note that the previous
feature was considered experimental as stated in d888d3c54c4744d52b9d21d3728f5d6dbc132cde
and the new feature should be considered experimental as well. The code
may be moved into a plugin in the future.
Followup 2f2da7274732cba30d03b6c5c3a4194652cb6783
This commit moves topic view tracking from happening
every time a Topic is requested, which is susceptible
to inflating numbers of views from web crawlers, to
our request tracker middleware.
In this new location, topic views are only tracked when
the following headers are sent:
* HTTP_DISCOURSE_TRACK_VIEW - This is sent on every page navigation when
clicking around the ember app. We count these as browser page views
because we know it comes from the AJAX call in our app. The topic ID
is extracted from HTTP_DISCOURSE_TRACK_VIEW_TOPIC_ID
* HTTP_DISCOURSE_DEFERRED_TRACK_VIEW - Sent when MessageBus initializes
after first loading the page to count the initial page load view. The
topic ID is extracted from HTTP_DISCOURSE_DEFERRED_TRACK_VIEW.
This will bring topic views more in line with the change we
made to page views in the referenced commit and result in
more realistic topic view counts.
