Commit Graph

540 Commits

Author SHA1 Message Date
1411b095eb DEV: Rescue errors when runngin AR failover/fallback callbacks. 2020-06-11 13:02:42 +08:00
34ee1f2e71 DEV: Fix undefined method in rails_failover initializer. 2020-06-11 12:30:01 +08:00
cbb4ea1ea8 DEV: Don't blow up request redis can't be reached during failover. 2020-06-11 11:41:06 +08:00
a3dfd553a1 Revert "Bump redis to 4.2.0."
This reverts commit 98bc28cea259b823552e1b953506bcf2f76347a9.
2020-06-10 14:52:05 +08:00
98bc28cea2 Bump redis to 4.2.0. 2020-06-10 14:28:56 +08:00
2ce829cc01 DEV: Allow multisite apps to boot with reading connection handler. 2020-06-10 14:18:29 +08:00
a1c13eb3c6 DEV: Redis failover should only clear redis recently readonly. 2020-06-09 16:36:31 +08:00
f6628e4f43 DEV: Disable messageBus keepalive when Redis fails over. 2020-06-08 12:33:08 +08:00
e82d4d8a75 DEV: Update rails_failover to avoid monkey patching Rails config. 2020-06-05 09:05:19 +08:00
87673e6571 DEV: Fix moving of rails_failover middleware. 2020-06-04 20:29:47 +08:00
aaece34e8b DEV: Update rails_failover so that we can move middleware up the stack. 2020-06-04 17:14:13 +08:00
54f79ea3ba DEV: Pausing Sidekiq is multisite aware. 2020-06-04 15:46:30 +08:00
8e1681d356 Bump rails_failover. 2020-06-04 15:22:35 +08:00
439db7ca1e DEV: Add REDIS_RAILS_FAILOVER env to test our new redis failover. 2020-06-02 17:24:14 +08:00
ade60b0cbc DEV: Enable readonly mode for all multisite sites when PG goes down.
The risk here is that the database for one site goes down in the multisite setup and we drop everything to readonly mode. However, I discussed this with Sam and we agree that one database having problem is very rare. Most of the time, it is the entire DB cluster that goes down.
2020-06-02 11:32:07 +08:00
326d6d5b0f DEV: Pause Sidekiq when forcing pg readonly mode. 2020-06-02 09:20:03 +08:00
32735be5bd DEV: Fix publish to message_bus when forcing pg readonly. 2020-06-01 13:26:12 +08:00
2c3ff3e524 DEV: Missing readonly mode banner when forcing PG readonly. 2020-06-01 12:25:27 +08:00
b0b37bf5a3 DEV: Add force rails_over switch via Redis. 2020-06-01 11:23:58 +08:00
8c86a109bb DEV: Add ENV flag to test out ActiveRecord::Failover. 2020-05-28 16:24:22 +08:00
878f06f1fe DEV: Remove custom connection reaper.
Rails 6 fixed the reaper to use one thread to reap all the connection pools.
2020-05-26 09:09:46 +08:00
d9a02d1336 Revert "Revert "Merge branch 'master' of https://github.com/discourse/discourse""
This reverts commit 20780a1eeed56b321daf18ee6bbfe681a51d1bf4.

* SECURITY: re-adds accidentally reverted commit:
  03d26cd6: ensure embed_url contains valid http(s) uri
* when the merge commit e62a85cf was reverted, git chose the 2660c2e2 parent to land on
  instead of the 03d26cd6 parent (which contains security fixes)
2020-05-23 00:56:13 -04:00
20780a1eee Revert "Merge branch 'master' of https://github.com/discourse/discourse"
This reverts commit e62a85cf6fd81a2a34aff6144bd36b9ac459964a, reversing
changes made to 2660c2e21d84bea667e1ea339f91cda352328062.
2020-05-22 20:25:56 -07:00
2f03a879f9 DEV: Require rails_failover before global settings. 2020-05-20 16:06:40 +08:00
f7f436e536 DEV: Install rails_failover gem to test our Redis changes. 2020-05-20 15:40:27 +08:00
96c02caba7 DEV: Change use of Redis flushall to flushdb.
FLUSHALL removes all keys from all databases. Instead we only want to
remove keys from the current Redis database.
2020-05-19 10:20:00 +08:00
609e929186 Revert "Revert "DEV: upgrade to Rails 6.0.3""
This reverts commit 2ff8b4f5d9dd72a55d7ebdc23a3758dd5ec4682d.

Attempt #2 at a Rails update this time we also update the
rails_multisite gem to allow for cleaner reordering
2020-05-08 11:49:22 +10:00
2ff8b4f5d9 Revert "DEV: upgrade to Rails 6.0.3"
This was causing issues during multisite:migrate

https://meta.discourse.org/t/multisite-migrate-broken-since-rails-6-0-3-update/150691

This reverts commit 136a5456534c7e468b8052f865ce970a3bf375e9.
2020-05-07 11:44:39 +01:00
136a545653 DEV: upgrade to Rails 6.0.3
Upgrades Rails to latest, this version has better compatibility
with Ruby 2.7

During the upgrade we needed a new cleaner mechanism for configuring
message bus.

All tests are green.

If anything weird pops up please revert.
2020-05-07 15:53:40 +10:00
d0d5a138c3 DEV: stop freezing frozen strings
We have the `# frozen_string_literal: true` comment on all our
files. This means all string literals are frozen. There is no need
to call #freeze on any literals.

For files with `# frozen_string_literal: true`

```
puts %w{a b}[0].frozen?
=> true

puts "hi".frozen?
=> true

puts "a #{1} b".frozen?
=> true

puts ("a " + "b").frozen?
=> false

puts (-("a " + "b")).frozen?
=> true
```

For more details see: https://samsaffron.com/archive/2018/02/16/reducing-string-duplication-in-ruby
2020-04-30 16:48:53 +10:00
17cf300b71 DEV: Use more specific error responses (#9472)
* DEV: Use `render_json_error` (Adds specs for Admin::GroupsController)
* DEV: Use a specific error on blank category slug (Fixes a `render_json_error` warning)
* DEV: Use a specific error on reviewable claim conflict (Fixes a `render_json_error` warning)
* DEV: Use specific errors in Admin::UsersController (Fixes `render_json_error` warnings)
* FIX: PublishedPages error responses
* FIX: TopicsController error responses (There was an issue of two separate `Topic` instances for the same record. This makes sure there's only one up-to-date instance.)
2020-04-21 03:50:20 +02:00
3d9c320aab PERF: Cache Category.subcategory_ids (#9350)
Also reset category cache after backup restore.
2020-04-09 15:42:24 +03:00
138d4aebde FEATURE: Webhooks and Event for user being granted a badge
Adding a webhook for badge revocation is left for future work as it's relatively rare.
2020-04-08 14:38:30 -07:00
bca126f3f5 REFACTOR: Move the multisite middleware to the front
Both request tracking and message bus rely on multisite before the
middleware has run which is not ideal.

Follow-up-to: ca1208a63669d4d4ad7452367008d40fa090f645
2020-04-02 16:44:44 +01:00
ca1208a636 Revert "REFACTOR: Move the multisite middleware to the front"
Looks like this is causing problems.

Follow-up-to: a91843f0dc7b97e700dc85505404eafd62b7f8c5
2020-04-02 15:20:28 +01:00
a91843f0dc REFACTOR: Move the multisite middleware to the front
Both request tracking and message bus rely on multisite before the
middleware has run which is not ideal.
2020-04-02 10:15:38 +01:00
25f1f23288 FEATURE: Stricter rules for user presence
Previously we would consider a user "present" and "last seen" if the
browser window was visible.

This has many edge cases, you could be considered present and around for
days just by having a window open and no screensaver on.

Instead we now also check that you either clicked, transitioned around app
or scrolled the page in the last minute in combination with window
visibility

This will lead to more reliable notifications via email and reduce load of
message bus for cases where a user walks away from the terminal
2020-03-26 17:36:52 +11:00
097851c135 FIX: Change secure media to encompass attachments as well (#9271)
If the “secure media” site setting is enabled then ALL files uploaded to Discourse (images, video, audio, pdf, txt, zip etc. etc.) will follow the secure media rules. The “prevent anons from downloading files” setting will no longer have any bearing on upload security. Basically, the feature will more appropriately be called “secure uploads” instead of “secure media”.

This is being done because there are communities out there that would like all attachments and media to be secure based on category rules but still allow anonymous users to download attachments in public places, which is not possible in the current arrangement.
2020-03-26 07:16:02 +10:00
fb8b0ca197 DEV: Remove unused middleware (#9203)
This has not been used since January 2014, per 166a8d29
2020-03-16 12:37:43 +00:00
8022e51179 FIX: Failed to restore backups from versions without translation overrides
Rails calls I18n.translate during initialization and by default translation overrides are used. Database migrations would fail if the system tried to migrate from an old version that didn't have the `translation_overrides` table with all its columns yet.

This makes restoring really old backups work again. Running `DISABLE_TRANSLATION_OVERRIDES=1 rake db:migrate` will allow you to upgrade such an old database as well.
2020-03-14 00:00:22 +01:00
a3f0543f99 Support for transpiling .js files (#9160)
* Remove some `.es6` from comments where it does not matter

* Use a post processor for transpilation

This will allow us to eventually use the directory structure to
transpile rather than the extension.

* FIX: Some errors and clean up in confirm-new-email

It would throw an error if the webauthn element wasn't present.
Also I changed things so that no-module is not explicitly
referenced.

* Remove `no-module`

Instead we allow a magic comment: `// discourse-skip-module` to prevent
the asset pipeline from creating a module.

* DEV: Enable babel transpilation based on directory

If it's in `app/assets/javascripts/dicourse` it will be transpiled
even without the `.es6` extension.

* REFACTOR: Remove Tilt/ES6ModuleTranspiler
2020-03-11 09:43:55 -04:00
537f87562e FIX: We need to skip users with associated reviewables when auto-approving (#9080)
* FIX: We need to skip users with associated reviewables when auto-approving them

* Update spec/initializers/track_setting_changes_spec.rb

* Update spec/initializers/track_setting_changes_spec.rb

Co-authored-by: Robin Ward <robin.ward@gmail.com>
2020-03-02 14:33:52 -05:00
14adddd18d FIX: Ignore secure-media-uploads for miniprofiler (#9070) 2020-02-28 12:11:30 +10:00
377d2d3fad DEV: Silence spurious rubocop lint warning 2020-02-19 13:10:30 +11:00
a14a7f1cb8 DEV: Add optional support for running byebug when a PG Clash happens
Tracking down concurrency issues from backtraces and manual repros is a fraught process.
Sometimes you've just got to get your hands dirty and do a live debug.
2020-02-19 12:50:37 +11:00
0b09f5299d DEV: Improve pg connection access logging
`ensure` that the accessing thread is set to nil after an action
2020-02-18 16:58:47 +00:00
ea49ca7ef5 DEV: Handle nil backtraces in pg access logs 2020-02-18 15:45:44 +00:00
2bdd1275ce DEV: Initialize pg access log mutex in non-sidekiq processes
Followup to be3e4ab3f55afb1f89bdd702f069b44a8bcef8e7
2020-02-18 14:20:28 +00:00
be3e4ab3f5 DEV: Report simultaneous use of PG::Connection objects 2020-02-18 13:50:15 +00:00
28292d2759 PERF: avoid shelling to get hostname aggressively
Previously we had many places in the app that called `hostname` to get
hostname of a server. This commit replaces the pattern in 2 ways

1. We cache the result in `Discourse.os_hostname` so it is only ever called once

2. We prefer to use Socket.gethostname which avoids making a shell command

This improves performance as we are not spawning hostname processes throughout
the app lifetime
2020-02-18 15:13:19 +11:00