Commit Graph

69 Commits

Author SHA1 Message Date
b3e52f99e6 FIX: LOAD_PLUGINS=0 in dev/prod, warn in plugin:pull_compatible_all (#15537)
The `plugin:pull_compatible_all` task is intended to take incompatible plugins and downgrade them to an earlier version. Problem is, when running the rake task in development/production environments, the plugins have already been activated. If an incompatible plugin raises an error in `plugin.rb` then the rake task will be unable to start.

This commit centralises our LOAD_PLUGINS detection, adds support for LOAD_PLUGINS=0 in dev/prod, and adds a warning to `plugin:pull_compatible_all` if it's run with plugins enabled.
2022-01-11 12:30:22 +00:00
c5fd8c42db DEV: Fix methods removed in Ruby 3.2 (#15459)
* File.exists? is deprecated and removed in Ruby 3.2 in favor of
File.exist?
* Dir.exists? is deprecated and removed in Ruby 3.2 in favor of
Dir.exist?
2022-01-05 18:45:08 +01:00
cfc62dbace FIX: allowed_theme_ids should not be persisted in GlobalSettings (#14756)
* FIX: allowed_theme_ids should not be persisted in GlobalSettings

It was observed that the memoized value of `GlobalSetting.allowed_theme_ids` would be persisted across requests, which could lead to unpredictable/undesired behaviours in a multisite environment.

This change moves that logic out of GlobalSettings so that the returned theme IDs are correct for the current site.

Uses get_set_cache, which ultimately uses DistributedCache, which will take care of multisite issues for us.
2021-10-29 11:46:52 -04:00
dda1cd6a38 FEATURE: allow setting postgres connection variables via environment 2021-04-14 13:31:32 -04:00
f84ff26aa9 FIX: use Redis replica host and port (#11566)
Introduce Redis `replica` config and deprecate `slave`
2020-12-23 13:14:19 +11:00
07bf7a91f4 Revert "FIX: use Redis replica host and port (#11543)" (#11552)
This reverts commit b0e1210b0cbd96ff0aa1669c7f8393b8340991e8.
2020-12-22 16:16:50 +11:00
b0e1210b0c FIX: use Redis replica host and port (#11543)
* FIX: use Redis replica host and port

Introduce Redis `replica` config and deprecate `slave`

* FIX: move deprecations to separate file
2020-12-22 15:52:00 +11:00
6e69c25134 FIX: Use asset_hostnames in rails_multisite
We want to respond to the x-forwarded-for host from the CDN too.
2020-09-17 10:32:43 +01:00
e0d9232259 FIX: use allowlist and blocklist terminology (#10209)
This is a PR of the renaming whitelist to allowlist and blacklist to the blocklist.
2020-07-27 10:23:54 +10:00
402b80f306 DEV: Make rails_failover compatible with SKIP_DB_AND_REDIS env. 2020-06-15 16:23:24 +08:00
58e52c0e4f DEV: Use rails_failover gem for ActiveRecord and Redis failover handling 2020-06-15 15:47:07 +08:00
d8cd912769 DEV: Switch to db config to disable advisory locks. 2020-06-15 14:33:41 +08:00
edc940ce3e DEV: Fix incorrect messagebus redis config for rails_failover. 2020-06-09 17:05:14 +08:00
57a3d4e0d2 FEATURE: whitelist theme repo mode (experimental)
In some restricted setups all JS payloads need tight control.

This setting bans admins from making changes to JS on the site and
requires all themes be whitelisted to be used.

There are edge cases we still need to work through in this mode
hence this is still not supported in production and experimental.

Use an example like this to enable:

`DISCOURSE_WHITELISTED_THEME_REPOS="https://repo.com/repo.git,https://repo.com/repo2.git"`

By default this feature is not enabled and no changes are made.

One exception is that default theme id was missing a security check
this was added for correctness.
2020-06-03 13:19:57 +10:00
439db7ca1e DEV: Add REDIS_RAILS_FAILOVER env to test our new redis failover. 2020-06-02 17:24:14 +08:00
8c86a109bb DEV: Add ENV flag to test out ActiveRecord::Failover. 2020-05-28 16:24:22 +08:00
878f06f1fe DEV: Remove custom connection reaper.
Rails 6 fixed the reaper to use one thread to reap all the connection pools.
2020-05-26 09:09:46 +08:00
d9a02d1336 Revert "Revert "Merge branch 'master' of https://github.com/discourse/discourse""
This reverts commit 20780a1eeed56b321daf18ee6bbfe681a51d1bf4.

* SECURITY: re-adds accidentally reverted commit:
  03d26cd6: ensure embed_url contains valid http(s) uri
* when the merge commit e62a85cf was reverted, git chose the 2660c2e2 parent to land on
  instead of the 03d26cd6 parent (which contains security fixes)
2020-05-23 00:56:13 -04:00
20780a1eee Revert "Merge branch 'master' of https://github.com/discourse/discourse"
This reverts commit e62a85cf6fd81a2a34aff6144bd36b9ac459964a, reversing
changes made to 2660c2e21d84bea667e1ea339f91cda352328062.
2020-05-22 20:25:56 -07:00
f7f436e536 DEV: Install rails_failover gem to test our Redis changes. 2020-05-20 15:40:27 +08:00
497dc6eaa7 Add a global setting for CDN origin
This is so that, on a multisite cluster, when we handle a CDN request,
the hostname that is requested corresponds to one of the sites -
specifically the default site.
2020-05-12 16:43:40 +01:00
54f67661ac FEATURE: Option to connect to Redis using SSL 2020-03-05 20:49:05 -03:00
0d3d2c43a0 DEV: s/\$redis/Discourse\.redis (#8431)
This commit also adds a rubocop rule to prevent global variables.
2019-12-03 10:05:53 +01:00
Sam
fa2a5f6f56 FEATURE: SKIP_DB_AND_REDIS env var (#7756)
Sometimes we would like to create a base image without any DB access, this
assists in creating custom base images with custom plugins that already
includes `public/assets`

Following this change set you can run:

```
SPROCKETS_CONCURRENT=1 DONT_PRECOMPILE_CSS=1 SKIP_DB_AND_REDIS=1 RAILS_ENV=production bin/rake assets:precompile
```

Then it is straight forward to create a base image without needing a DB or
Redis.
2019-06-13 12:58:27 +10:00
315a38e0e3 FEATURE: Allow running message_bus in a different redis instance (#7616)
Adds `DISCOURSE_MESSAGE_BUS_REDIS_ENABLED` env var, that when set
to true, will allow Discourse to connect to a different redis
instance for MessageBus needs.

When enabled you can configure the same env vars user for redis,
but prefixed by `MESSAGE_BUS`, eg:

`DISCOURSE_MESSAGE_BUS_REDIS_HOST`
2019-05-28 15:52:43 +10:00
30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
152238b4cf DEV: Prefer public_send over send. 2019-05-07 09:33:21 +08:00
b375dcb14a DEV: Introduce parallel rspec testing
Adds the parallel_tests gem, and redis/postgres configuration for running rspec tests in parallel. To use:

```
rake parallel:rake[db:create]
rake parallel:rake[db:migrate]
rake parallel:spec
```

This brings the test suite from 12m20s to 3m11s on my macOS machine
2019-04-01 11:06:47 -04:00
Sam
8b7a2d1cb7 FEATURE: add setting to bypass sending redis CLIENT commands
Some cloud providers (Google Memorystore) do not support any CLIENT commands

By setting :id to nil in the redis config hash we can avoid these commands.

This adds a special global setting GCE users can enable:
`DISCOURSE_REDIS_SKIP_CLIENT_COMMANDS = true`
2019-01-04 15:08:33 +11:00
dba22bbde2 rollback changes
This reverts:
* 1baba84c438e "fix s3 subfolders harder"
* ea5e57938edf "fix test for absolute_base_url change"
2018-07-06 17:16:40 -05:00
52e9f49ec1 fix s3 subfolders harder
specifically, include the folder in absolute_base_url
2018-07-06 16:28:40 -05:00
bc89674b58 FIX for bucket name in GlobalSetting with folder
When `s3_bucket="bucket/folder` in discourse.conf, absolute_base_url
was bucket/folder.s3-region.amazonaws.com

These names are bad, but this mirrors the s3_bucket/s3_bucket_name in
S3Store

N.B. that nearby s3_upload_bucket _should_ include the folder
2018-05-17 12:59:34 -05:00
976d6b290c FIX: CDN_URL hostname should be in GlobalSetting.hostnames 2018-03-27 15:20:22 -04:00
a89f3160a5 Add new config to ensure backup/restore connects to PG directly.
* In `pg_dump` 10.3+ and 9.5.12+, in
  it does a `SELECT pg_catalog.set_config('search_path', '', false)`
  which changes the state of the current connection. This is known
  to be problematic with Pgbouncer which reuses connections. As such,
  we'll always try to connect directly to PG directly during
  the backup/restore process.
2018-03-09 10:28:03 +08:00
Sam
20fbf81505 FEATURE: Allow plugins to easily add defaults to GlobalSetting
usage:

GlobalSetting.add_default("plugin_setting_name", "value")
2017-11-16 08:39:29 +11:00
b54eb8f53c FIX: Set PG connect_timeout to 5 seconds.
* 30 seconds is alittle too long.
2017-10-17 12:32:41 +08:00
Sam
4ea87b5ab8 Merge branch 's3_refactor' 2017-10-09 10:27:52 +11:00
Sam
70bb2aa426 FEATURE: allow specifying s3 config via globals
This refactors handling of s3 so it can be specified via GlobalSetting

This means that in a multisite environment you can configure s3 uploads
without actual sites knowing credentials in s3

It is a critical setting for situations where assets are mirrored to s3.
2017-10-06 16:20:01 +11:00
5dc4b469be Remove unused code for Redis Sentinel. 2017-10-05 14:53:03 +08:00
9bc3038728 Fix Redis command errors when trying to start app with a readonly Redis. 2017-08-02 21:33:56 +09:00
5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
a916f6b0c8 remove some warnings with already declared constants 2017-06-22 12:55:58 +02:00
Sam
64680286f4 correct logic, so revalidation is reset
correct test so it can run at any point
2017-03-13 10:47:43 -04:00
Sam
ef24fd54ba FEATUE: automatically validate token is stored in redis
This ensures we have some handling for redis flushall

We attempt to recover our in-memory session token once every 30 seconds

Code is careful to only set the token if it is nil, to allow for manual
cycling to remain safe if needed
2017-03-13 10:19:02 -04:00
Sam
3032aa7db9 PERF: avoid looking globals from providers after first call 2017-03-09 18:00:55 -05:00
Sam
ff49f72ad9 FEATURE: per client user tokens
Revamped system for managing authentication tokens.

- Every user has 1 token per client (web browser)
- Tokens are rotated every 10 minutes

New system migrates the old tokens to "legacy" tokens,
so users still remain logged on.

Also introduces weekly job to expire old auth tokens.
2017-02-07 09:22:16 -05:00
3c28d94706 FIX: Don't configure Redis connector if Redis slave config is not set. 2017-02-02 13:48:55 +08:00
b60bc47a4c Plugins can register providers for global settings 2017-01-09 17:18:58 -05:00
c07c474575 FEATURE: Master-Slave Redis configuration with fallback and switch over. 2016-03-11 12:18:58 +08:00
46589a1a0c FEATURE: AR adapter to failover to a replica DB server. 2016-02-05 08:51:10 +08:00