Commit Graph

257 Commits

Author SHA1 Message Date
2717615f11 PERF: Preload client settings json while booting app. (#13283)
Generating the client settings json involves santizing all string based
site settings. This is expensive as per our profile in production (~120ms) and one request after each deploy has
to pay this penalty.
2021-06-08 13:15:55 +10:00
cadf5eafe6 DEV: Move Discourse app specific concern out of unicorn conf. 2021-06-04 09:13:34 +08:00
58b30fb510 PERF: Preload settings, groups and badge icons in SvgSprite.
Identified as a hot path in production. Preload it early instead of
executing the queries in a live request.
2021-06-03 16:45:56 +08:00
2e4182b4b3 PERF: Warm up caches in SiteSerializer while preloading app.
Otherwise, time will be spent warming up the caches in the initial
requests.
2021-06-02 13:33:31 +08:00
0700e9382d PERF: Memoize core svgs in memory to avoid expensive XML parsing.
The XML parsing of SVGs is done whenever the cache expires or on the
first load after a reboot. In one of our production instance, parsing
ranges from 30ms to 70ms which is not ideal. Instead, we've decided to
make a small memory trade off here by memoizing the core SVGs once on
boot to avoid parsing of the SVG files during the duration of a request.
The memozied hash will take up 57440 bytes or 0.05744 megabytes in size.
2021-06-02 09:16:37 +08:00
c1c090419b DEV: Fix warning about already initialized constants in FileStore::BaseStore 2021-06-01 09:30:37 +08:00
f05f5fde0f PERF: Preload JsLocaleHelper.load_translations for default locale. (#13041)
In production, each Unicorn child process will currently hold the
default locale in memory on first load. Instead, we should preload it in
the Unicorn master process so that the memory is immediately shared when
forking.

Also, the translations are only memoized on first load now and is
adding considerable overhead to the first few requests after a fresh
boot.
2021-05-13 16:16:01 +10:00
4bad1e7e93 PERF: Preload LetterAvatar.image_magick_version in master process.
It is wasteful to run the clean up and command to generate the version
digest on every unicorn child process.
2021-05-07 14:00:38 +08:00
cac7725e28 DEV: Improve bin/unicorn boot time in development environment (#12900)
This commit makes a few changes to improve boot time in development environments. It will have no effect on production boot times.

- Skip the SchemaCache warmup. In development mode, the SchemaCache is refreshed every time there is a code change, so warmup is of limited use.

- Skip warming up PrettyText. This adds ~2s to each web worker's boot time. The vast majority of requests do not use PrettyText, so it is more efficient to defer its warmup until it's needed

- Skip the intentional 1 second pause during Unicorn worker forking. The comment (which also exists in Unicorn's documentation) suggests this works around a Unix signal handling bug, but I haven't been able to locate any more information. Skipping it in dev will significantly speed up boot. If we start to see issues, we can revert this change.

On my machine, this improves `/bin/unicorn` boot time from >10s to ~4s
2021-04-30 11:32:13 +01:00
e3b1d1a718 DEV: Improve Ember CLI's bootstrap logic (#12792)
* DEV: Give a nicer error when `--proxy` argument is missing

* DEV: Improve Ember CLI's bootstrap logic

Instead of having Ember CLI know which URLs to proxy or not, have it try
the URL with a special header `HTTP_X_DISCOURSE_EMBER_CLI`. If present,
and Discourse thinks we should bootstrap the application, it will
instead stop rendering and return a HTTP HEAD with a response header
telling Ember CLI to bootstrap.

In other words, any time Rails would otherwise serve up the HTML for the
Ember app, it stops and says "no, you do it."

* DEV: Support asset filters by path using a new options object

Without this, Ember CLI's bootstrap would not get the assets it wants
because the path it was requesting was different than the browser path.
This adds an optional request header to fix it.

So far this is only used by the styleguide.
2021-04-23 10:24:42 -04:00
0ec5fd5262 DEV: Raise exception when execute_command will spawn a shell (#12716) 2021-04-15 16:29:37 +01:00
65647000a1 DEV: Allow Discourse::Utils.execute_command timeout with env (#12672)
Followup to 5deda5ef3ef8763dfe936a313e0dddaa279a1b04

The first argument to `Open3.capture3` can be an environment variable hash. In this case, we need to insert the `timeout` command after the env hash.
2021-04-12 13:53:41 +01:00
Sam
5deda5ef3e FIX: automatically timeout long running image magick commands (#12670)
Previously certain images may lead to convert / identify to run for unreasonable
amounts of time

This adds a maximum amount of time these commands can run prior to forcing
them to stop
2021-04-12 13:55:54 +10:00
a5923ad603 DEV: apply allow origin response header for CDN requests. (#11893)
Currently, it creates a CORS error while accessing those static files.
2021-01-29 07:44:49 +05:30
123107c28f UX: Add group name to error message (#11333)
The group name used to be part of the error message, but was removed
in a past commit.
2020-11-24 13:06:52 +02:00
9b7525bb03 FIX: Handle uncaught exception (#11263)
After the search term is parsed for advanced search filters, the term may
become empty. Later, the same term will be passed to Discourse.route_for
which will raise an ArgumentError.

> URI(nil)
ArgumentError: bad argument (expected URI object or URI string)
2020-11-20 11:28:14 +02:00
5289fc7886 FIX: Improve failover for multisite clusters (#11150)
- Bump rails_failover for new per-backend callback feature
- If the master backend fails over, make all sites readonly. And vice-versa for fallback
- If a single backend fails over, make that individual site readonly. And vice-versa for fallback
- When a single backend fails, also check connection to the master backend
2020-11-11 10:27:24 +00:00
af4938baf1 Revert "DEV: enable cors to all cdn get requests from workbox. (#10684)" (#11076)
This reverts commit e3de45359f9bc7a71d9b8045a7e369fd0cf8f433.

We need to improve out strategy by adding a cache breaker with this change ... some assets on CDNs and clients may have incorrect CORS headers which can cause stuff to break.
2020-10-30 16:05:35 +11:00
347423007a DEV: remove instagram login site settings and auth classes. (#11073)
Instagram removed the support for login and should use Facebook login instead.
2020-10-30 09:09:56 +05:30
e3de45359f DEV: enable cors to all cdn get requests from workbox. (#10685)
Now all external requests from the service worker will be in CORS mode without credentials.
2020-10-28 23:36:19 +05:30
721ee36425 Replace base_uri with base_path (#10879)
DEV: Replace instances of Discourse.base_uri with Discourse.base_path

This is clearer because the base_uri is actually just a path prefix. This continues the work started in 555f467.
2020-10-09 12:51:24 +01:00
c5c5bb4fad DEV: prevent already initialized constant warning 2020-08-07 16:28:43 +02:00
255b0e9f14 PERF: Replace video and audio links in search blurb while indexing.
In the near future, we will be swtiching to PG headlines to generate the
search blurb. As such, we need to replace audio and video links in the
raw data used for headline generation. This also means that we avoid
replacing links each time we need to generate the blurb.
2020-08-06 12:25:03 +08:00
0884d570b1 FEATURE: add support for top filter in tag page. (#10281)
Currently, tag pages only have the `latest` filter.
2020-07-22 19:26:36 +05:30
94a2a70462 DEV: Use a longer TTL for pg readonly mode. 2020-07-14 16:15:58 +08:00
58e52c0e4f DEV: Use rails_failover gem for ActiveRecord and Redis failover handling 2020-06-15 15:47:07 +08:00
0ff86b00cb DEV: Upgrade Redis to 4.2.1. 2020-06-15 10:05:22 +08:00
78b5ab746c DEV: No longer need to clear anon cache when toggling readonly mode. 2020-06-12 09:58:17 +08:00
90dc6c8ce5 DEV: Fix deprecation warning when using Redis#close with redis namespace. 2020-06-11 14:09:19 +08:00
531eca985e DEV: Log error backtrace in Discourse.warn_exception without logster. 2020-06-11 10:49:46 +08:00
a3dfd553a1 Revert "Bump redis to 4.2.0."
This reverts commit 98bc28cea259b823552e1b953506bcf2f76347a9.
2020-06-10 14:52:05 +08:00
98bc28cea2 Bump redis to 4.2.0. 2020-06-10 14:28:56 +08:00
a1c13eb3c6 DEV: Redis failover should only clear redis recently readonly. 2020-06-09 16:36:31 +08:00
8e1681d356 Bump rails_failover. 2020-06-04 15:22:35 +08:00
76ce814a62 DEV: Use safe_each_connection patch when preloading Rails.
This is to ensure we still boot even if there is a problem with any of
the databases.
2020-06-03 15:36:50 +08:00
fe9c82994d DEV: Actually disconnect from Redis connections after fork. 2020-06-02 11:40:16 +08:00
df62407f35 DEV: Implement multiple keys support for DiscourseRedis#exists. 2020-06-01 11:20:26 +08:00
3894555b2e DEV: Prefer Discourse.redis.reconnct. 2020-06-01 10:55:53 +08:00
3b311f6b15 DEV: Bump rails_failover. 2020-05-29 09:02:34 +08:00
878f06f1fe DEV: Remove custom connection reaper.
Rails 6 fixed the reaper to use one thread to reap all the connection pools.
2020-05-26 09:09:46 +08:00
09caf614de DEV: Fix build
Rails.logger is not available when Discourse.deprecate is called from
enabled_site_setting_filter.

Follow up to db4601823546b7a1451b82a6f556a30a55166a21.
2020-05-10 15:06:39 +03:00
8149bfbaf1 FEATURE: Filter settings by plugin (#9692) 2020-05-10 14:07:45 +03:00
d0d5a138c3 DEV: stop freezing frozen strings
We have the `# frozen_string_literal: true` comment on all our
files. This means all string literals are frozen. There is no need
to call #freeze on any literals.

For files with `# frozen_string_literal: true`

```
puts %w{a b}[0].frozen?
=> true

puts "hi".frozen?
=> true

puts "a #{1} b".frozen?
=> true

puts ("a " + "b").frozen?
=> false

puts (-("a " + "b")).frozen?
=> true
```

For more details see: https://samsaffron.com/archive/2018/02/16/reducing-string-duplication-in-ruby
2020-04-30 16:48:53 +10:00
3d71b68195 DEV: Introduce plugin api for conditionally rendering assets (#9200) 2020-03-13 15:30:31 +00:00
a3f0543f99 Support for transpiling .js files (#9160)
* Remove some `.es6` from comments where it does not matter

* Use a post processor for transpilation

This will allow us to eventually use the directory structure to
transpile rather than the extension.

* FIX: Some errors and clean up in confirm-new-email

It would throw an error if the webauthn element wasn't present.
Also I changed things so that no-module is not explicitly
referenced.

* Remove `no-module`

Instead we allow a magic comment: `// discourse-skip-module` to prevent
the asset pipeline from creating a module.

* DEV: Enable babel transpilation based on directory

If it's in `app/assets/javascripts/dicourse` it will be transpiled
even without the `.es6` extension.

* REFACTOR: Remove Tilt/ES6ModuleTranspiler
2020-03-11 09:43:55 -04:00
83e649d08e FIX: Clobbering plugin files on startup is problematic
On startup, (including when starting a rails console) we manipule a
collection of plugin files. Writing these files is done in multiple
observable steps, which presents opportunities for race conditions and
causes temporary corruption.

This commit uses the write, fsync and rename trick to atomically
overwrite these files instead, but reads them first to avoid unnecessary
writes.

c457d3bf was a previous attempt to fix the same problem.
2020-03-05 11:51:56 +00:00
28292d2759 PERF: avoid shelling to get hostname aggressively
Previously we had many places in the app that called `hostname` to get
hostname of a server. This commit replaces the pattern in 2 ways

1. We cache the result in `Discourse.os_hostname` so it is only ever called once

2. We prefer to use Socket.gethostname which avoids making a shell command

This improves performance as we are not spawning hostname processes throughout
the app lifetime
2020-02-18 15:13:19 +11:00
556d66a97c FIX: Use CDN for the discourse-internet-explorer
Previously the CDN was skipped because the assets were not in the proper
place to be uploaded.
2020-01-10 15:06:55 -05:00
3b7f5db5ba FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547) 2019-12-18 11:21:57 +05:30
0d3d2c43a0 DEV: s/\$redis/Discourse\.redis (#8431)
This commit also adds a rubocop rule to prevent global variables.
2019-12-03 10:05:53 +01:00