This PR adds a destroy:posts rake task that can be used to hard-delete a list of posts. Useful for dealing with large amounts of spam that has been soft deleted and needs to go.
Notes:
Works on both non-deleted and soft-deleted posts. (We might want to change this to work on only soft-deleted posts?)
Works exclusively on post IDs. We can't mix topic and post IDs as they might clash, and we have no way of resolving that ambiguity.
Accepts either a rake-style array of IDs or, more conveniently, you can pipe the argument in through STDIN.
Added a confirmation step since it's a fairly destructive operation.
Currently, after creating an API key, there is no way in the UI to see what scope the key has. To do this we need to first store the selected scope mode when creating a new key.
In this PR we:
- Convert scope_mode from a transient attribute to a database backed enum.
- Ship the possible values through the javascript:update_constants rake task instead of hard coding in front-end.
In follow-up PRs we will:
- Backfill existing API keys based on their associated api_key_scopes records.
- Start showing the scope mode in the UI.
This commit moves most of emoji logic into the discourse-emojis gem:
https://github.com/discourse/discourse-emojis/
Most notably:
- images are now symlinked from the gem
- the gem provides path to the json files
Search aliases have also been made asynchronous and memoized. When you
will search for an emoji we will now load the aliases and store the list
for future use.
---------
Co-authored-by: David Taylor <david@taylorhq.com>
Followup e26a1175d7c33746bddbc858ad89e68cc14beefe
Adds extra functionality and tests for the admin search modal.
* Show third level plugin config pages in search, e.g. AI Usage
* Remember last used search filters
* Allow navigating search results with keyboard, using tab or up/down
and enter to go to result
* Add a placeholder beneath search input to tell the admin what to do
* Add a full page search at `/admin/search` which can be reached from
pressing Enter on the search input
* Add specs for modal and full page search
* Change admin sidebar filter "no results found" to point to full page
search
* Add keyboard shortcut help to modal for admin search
This totally separate SCSS and i18n compilation pipelines only existed
so that we could run `ember exam` in CI without starting Rails.
Now that our CI has such heavy caching of Ruby dependencies and database
migrations, the speed benefit of this is not worth the cost of
maintaining these separate pipelines.
Therefore, this commit removes that system, and updates CI to use
`bin/rake qunit:test`. That will start up a Rails server and proxy
stylesheet/locale requests to it. This strategy was already used for our
theme and plugin qunit test runs.
This is a follow up to a9eefd1b4849573452581c113cfcbdd4420fb3c2.
### Reviewer notes
This rake task is quite annoying to test so I don't think the time
trying to figure out how to test it is worth it. This does not sit in a
critical path anyway so I think the lack of tests here is OK.
This change adds a new `type_source` field to the `Reviewable` model, indicating whether the Reviewable type was registered by `core`, a plugin, or an `unknown` source.
When a plugin that registered a Reviewable type is disabled, this allows us to tell the user which plugin they need to re-enable to handle any orphan reviewable items.
This commit contains a couple of improvements for this
rake task.
* We no longer limit the uploads to only ones with Post
upload references, it doesn't matter what the secure uploads
are linked to, they should all be un-secured
* We now only get distinct uploads from the initial query,
multiple upload references on the same upload caused
double ups and confusing counts for the task
* We now also disable the secure_uploads_pm_only site
setting at the same time
Rake allows env variables to specified in arguments, so we need to use
the list of top_level_tasks which excludes those env
Followup to c8718a64dd7a26165efe91706ce5507e6999044a.
By design, db:create initializes the Rails app with SKIP_DB=true. That
means that SiteSettings get set up with the LocalProcessProvider instead
of the DBProvider. In other words: any calls to site settings will
return the default, rather then the actual value in the database.
Running db:migrate in the same rake invocation means that rails will not
be re-initialized, and so skip_db will remain true. Site settings
accessed during migrations and fixtures will therefore return incorrect
values.
One example of this is that running bin/rake db:create db:migrate
repeatedly in a development environment will cause the FAQ topic to be
seeded repeatedly, because the seed logic does not have access to the
site setting which stores the already-seeded topic id.
This commit will automatically re-exec the Rake command if any tasks are
specified after `db:create`
This commit updates the `uploads:sync_s3_acls` rake task to accept a
`sync` argument that would run the rake task in a synchronous manner
so that the outcome of running the rake task can easily be determined.
This reverts commit b9f8a77d9ba5222bbf55172092f0b235b92d1c85.
Reverting while we work on adding error handling. At the moment,
failures are logged, but the process still exits with status=0.
This commit converts the `AdminReport` component, which is quite
high complexity, to gjs. After this initial round, ideally this
component would be broken up into smaller components because it is
getting quite big now.
Also in this commit:
* Add an option to display the report description in a tooltip, which
was
the main way the description was shown until recently. We want to use
this on the dashboard view mostly.
* Move admin report "mode" definitions to the server-side Report model,
inside a `Report::MODES` constant, collecting the modes defined in
various
places in the UI into one place
* Refactor report code to refer to mode definitions
* Add a `REPORT_MODES` constant in JS via javascript.rake and refactor
JS to refer to the modes
* Delete old admin report components that are no longer used
(trust-level-counts, counts, per-day-counts) which were replaced
by admin-report-counters a while ago
* Add a new `registerReportModeComponent` plugin API, some plugins
introduce their own modes (like AI's `emotion`) and components and
we need a way to render them
This is a revert of 92793c5b73871ba84b024c2ce50055a0776f1ba6.
Following on from discussions after the previous commit, it became evident that it was only a small step towards solving the larger problem of finding site settings in a reliable fashion across multiple languages.
This is going to take more thought and discussion, and since the changes introduced in the previous commit are effectively non functional without additional work, I'm going to revert it for now.
This commit introduces a new rake task that can be used in situations where a community receives a large number of flags/reports and needs a quick way to handle all of those pending reports. Usage instructions are included in the rake task source code.
Internal topic: t/145475.
The chat emoji picker is renamed emoji-picker, and the old emoji-picker is removed.
This commit doesn't attempt to fully rework a new emoji-picker but instead tries to migrate everything to one picker (the chat one) and add small changes.
Other notable changes:
- all the favorite emojis code has been mixed into one service which is able to store one state per context, favorites emojis will be stored for all topics, and for each chat channel. Meaning that if you always use a specific emoji in a channel, it will only show as favorite emoji in this channel.
- a lot of static code has been removed which should improve initial load perf of discourse. Initially this code was around to improve the performance of the emoji picker rendering.
- the emojis are now stored, once the full list has been loaded, if you close and reopen the picker it won't have to load them again.
List of components:
- `<EmojiPicker />` will render a button which will open a dropdown
- `<EmojiPickerContent />` represents the content of the dropdown alone, it's useful when you want to render a picker from an action which is not the default picker button
- `<EmojiPickerDetached />` just a simple wrapper over `<EmojiPickerContent />` to make it easier to use it with `this.menu.show(...)`
---------
Co-authored-by: Renato Atilio <renatoat@gmail.com>
Adds a new reviewables:populate rake task that works in a similar fashion to the existing *:populate rake tasks. The rake task creates pending reviewable of all core types, with possibility for plugins to extend the task to populate their own reviewable types.
This commit replaces the `full_name_required` setting with a new `full_name_requirement` setting to allow more flexibility with the name field in the signup form. The new setting has 2 options, "Required at signup" and "Optional at signup", which are equivalent to the true/false possibilities of the old setting, and a third option "Hidden at signup" that hides the name field from the signup form, making it effectively optional too.
New sites will have the "Hidden at signup" option as the default option, and existing site will continue to use the option that maps to their current configuration.
Internal topic: t/136746.
We can't delete the file from disk as some of the assets are still
served by the app instead of going through the S3 bucket. It is a bug we
need to fix but it also means this ENV is unsafe now. Just drop the env
until we ensure all assets requested by the app are requested from the
S3 bucket directly.
The `max_compress?` logic is totally broken at least when used for
brotli compression because we are only seeing 4 assets subjected to the
max compression level in production. Instead of fixing the broken logic,
we should just drop this unnecessary complexity cause things are easier
to reason about when we only have one compression level to deal with
across all assets.
Now that we run the `upload` method in different threads, we need to
synchronize writes to `STDOUT` which we can do so by using a `Logger`.
Follow-up to 49e835395966fbddb9ad2988961d8375e501151f
Following a recent commit (cb4b8146a30cd94b04db1b662fbebdb8021a7771),
the benchmark script wasn’t working anymore (and the related rake task).
This patch fixes it. It also adds some information about Ruby YJIT being
enabled or not.
It splits the hide_profile_and_presence user option and the default_hide_profile_and_presence site setting for more granular control. It keeps the option to hide the profile under /u/username/preferences/interface and adds the presence toggle in the quick user menu.
Co-authored-by: Régis Hanol <regis@hanol.fr>
This commit changes the uploads:secure_upload_analyse_and_update
and uploads:disable_secure_uploads to no longer rebake affected
posts inline. This just took way too long, and if the task stalled
you couldn't be sure if the rest of it completed.
Instead, we can update the baked_version of affected posts and
utilize our PeriodicalUpdates job to gradually rebake them. I added
warnings about increasing the site setting rebake_old_posts_count and
the global setting max_old_rebakes_per_15_minutes before doing this
as well.
For good measure, the affected post IDs are written to a JSON file too.
The `categories_only_optimized` category page style has been introduced
in commit d37a0d401ceca044002fd640be3745de643a3f2a. This commit makes
sure that style is enforced for users who can see over 1000 categories
in order to keep `/categories` page functional.
Followup 30fdd7738ec942126bb055c9a6a8d69ddffba2da
Adds a new site setting and corresponding user preference
to disable smart lists. By default they are enabled, because
this is a better experience for most users. A small number of
users would prefer to not have this enabled.
Smart lists automatically append new items to each
list started in the composer when enter is pressed. If
enter is pressed on an empty list item, it is cleared.
This setting will be removed when the new composer is complete.
- Add concurrency when running on multisite clusters (default 10, configurable via THEME_UPDATE_CONCURRENCY env)
- Add a version cache for the duration of the rake task. This avoids duplicating work when many sites in the cluster have the same theme installed, and it is already up-to-date
- Updates output to be more concurrent friendly (all `puts`, no `print`)
- limits security key deletes to second factor keys
- also deletes backup codes (lingering backup codes break login flow entirely)
* Add spec for rake task to disable 2FA for a user
Remove emoji.clear cache calls as data.js.es6.erb hasn't existed in a while.
Emoji data is now compiled separately via javascript rake tasks.
Skip db and redis precompilation when no db is present
We were running into errors running `ember build` on machines with high
CPU counts. It was then noted that `thread-loader`, which embroider uses, defaults to spinning
up x workers where x is number of physical CPU cores - 1. That is
probably too much so we set out to find out an optimial count to set for
the `JOBS` env which embroider will use to set the number of
`thread-loader` workers.
I first built an image using the following Dockerfile.
```
FROM discourse/base:release
RUN cd /var/www/discourse && sudo -EH -u discourse bundle exec rake plugin:install_all_official
RUN cd /var/www/discourse && sudo -EH -u discourse bundle exec rake assets:precompile:prereqs
```
I then ran the following command on my M3 Max Macbook Pro that has 14
phyisal CPU cores.
```
for j in 1 2 4 8 14; do echo "JOBS=$j"; time docker run --rm -it -e JOBS=$j test:latest /bin/bash -c "su discourse -c 'cd /var/www/discourse && bundle exec rake assets:precompile:build'"; done
```
These are the results I got:
```
JOBS=1 0.04s user 0.03s system 0% cpu 1:01.92 total
JOBS=2 0.04s user 0.02s system 0% cpu 42.605 total
JOBS=4 0.04s user 0.02s system 0% cpu 37.012 total
JOBS=8 0.04s user 0.02s system 0% cpu 35.199 total
JOBs=14 0.04s user 0.02s system 0% cpu 37.941 total
```
We think JOBS=2 is a good default when the `JOBS` env has not been set.
Anything above just consumes more resources for little benefit.
