Commit Graph

10391 Commits

Author SHA1 Message Date
eb8a224fd2 FIX: Improve error handling for messageformat compilation (#28015)
We are investigating reports of errors with messageformat compilation following 301713ef. This commit does not fix the issue, but it introduces some basic error handling to avoid completely breaking affected sites.

We will have a fix for the root cause soon.
2024-07-22 13:15:10 +01:00
fb7cc2d375 FIX: stop memoize PostActionTypes (#28005)
Memoizing all_flags on PostActionType was a mistake. This commit brings back the cache on the serialize level.
2024-07-22 17:35:49 +10:00
43aa47b118 FIX: experimental topics filter should allow tags with underscore (#27994)
When tags contain an underscore we should allow filtering in the same way, previously due to the regex those with underscores were not being found when filtering.
2024-07-20 00:58:29 +04:00
5038cad68e DEV: Restore missing_s3_uploads stats count if site was restored (#27984)
This commit ensures that we reset the `missing_s3_uploads` status count
if there are no inventory files which are at least 2 days older than the
site's restored date.

Otherwise, a site with missing uploads but was subsequntly restored will
be continue to report missing uploads for 2 days.
2024-07-19 14:22:58 +08:00
f5cbc3e3b8 FEATURE: Allow oneboxing private GitHub repo URLs and add private indicator to HTML (#27947)
Followup 560e8aff75e4bde67bb162e8fdea52e704a19f81

The linked commit allowed oneboxing private GitHub PRs,
issues, commits, and so on, but it didn't actually allow
oneboxing the root repo e.g https://github.com/discourse/discourse-reactions

We didn't have an engine for this, we were relying on OpenGraph
tags on the HTML rendering of the page like we do with other
oneboxes.

To fix this, we needed a new github engine for repos specifically.

Also, this commit adds a `data-github-private-repo` attribute to
PR, issue, and repo onebox HTML so we have an indicator of
whether the repo was private, which can be used for theme components
and so on.
2024-07-19 12:21:45 +10:00
4b20021033 DEV: Restrict include:unlisted search option to users that can view unlisted topics (#27977) 2024-07-18 16:33:14 -03:00
6a3e12a39c FEATURE: Include advanced search option to include unlisted topics in the results (#27958)
---------

Co-authored-by: Régis Hanol <regis@hanol.fr>
2024-07-18 13:43:53 -03:00
48d13cb231 UX: Use a dropdown for SSL mode for group SMTP (#27932)
Our old group SMTP SSL option was a checkbox,
but this was not ideal because there are actually
3 different ways SSL can be used when sending
SMTP:

* None
* SSL/TLS
* STARTTLS

We got around this before with specific overrides
for Gmail, but it's not flexible enough and now people
want to use other providers. It's best to be clear,
though it is a technical detail. We provide a way
to test the SMTP settings before saving them so there
should be little chance of messing this up.

This commit also converts GroupEmailSettings to a glimmer
component.
2024-07-18 10:33:14 +10:00
c975c7fe1b FEATURE: custom flag can require additional message (#27908)
Allow admin to create custom flag which requires an additional message.

I decided to rename the old `custom_flag` into `require_message` as it is more descriptive.
2024-07-18 10:10:22 +10:00
b3e0e920ed DEV: Support adding a custom filter on /filter (#27927)
# Context

Currently there is no way to add a custom filter to the experimental `/filter` endpoint. While you can implement a custom `status:` there is no way to include the user's input in a custom query. 

# PR

This PR adds the ability to implement a custom filter. eg. `CUSTOM_FILTER:foo`

- Add `add_filter_custom_filter` for extension
- Add specs
2024-07-17 11:36:38 -05:00
0783bfbbfe FIX: Use login SMTP auth for office365 in group mailer (#27931)
Followup 7b627dc14b934430f62a59fbcf09b0595ee94567

In this other commit, I changed the email settings validator
to always use the `login` authentication method for
office365 and outlook, but I didn't change the actual
group SMTP mailer to do this.

This commit fixes that issue and does some minor refactoring.
2024-07-16 16:21:14 +10:00
25778d9861 FIX: Return 400 response codes when topic list query params are invalid (#27930)
This commit updates `TopicQuery.validators` to cover all of the
public options listed in `TopicQuery.public_valid_options`. This is done
to fix the app returning a 500 response code when an invalid value, such
as a hash, is passed as a query param when accessing the various topic
list routes.
2024-07-16 10:30:04 +08:00
Nat
d32675ace2 Bump version to v3.3.0.beta5-dev 2024-07-15 20:30:15 +08:00
Nat
c9e24eca39 Bump version to v3.3.0.beta4 2024-07-15 20:30:14 +08:00
97e2b353f6 FEATURE: Allow for multiple GitHub onebox tokens (#27887)
Followup 560e8aff75e4bde67bb162e8fdea52e704a19f81

GitHub auth tokens cannot be made with permissions to
access multiple organisations. This is quite limiting.
This commit changes the site setting to be a "secret list"
type, which allows for a key/value mapping where the value
is treated like a password in the UI.

Now when a GitHub URL is requested for oneboxing, the
org name from the URL is used to determine which token
to use for the request.

Just in case anyone used the old site setting already,
there is a migration to create a `default` entry
with that token in the new list setting, and for
a period of time we will consider that token valid to
use for all GitHub oneboxes as well.
2024-07-15 13:07:36 +10:00
9e4e591d60 Revert "FEATURE: custom flag can require additional message (#27706)" (#27906)
This reverts commit c0bcd979e38134989b9023b75738caa743d2e092.
2024-07-15 09:45:57 +10:00
c0bcd979e3 FEATURE: custom flag can require additional message (#27706)
Allow admin to create custom flag which requires an additional message.

I decided to rename the old `custom_flag` into `require_message` as it is more descriptive.
2024-07-15 08:48:01 +10:00
8fe73ab404 DEV: better log around nil client_settings_json (#27868)
Internal ref t/132507
2024-07-11 11:30:53 +02:00
5ec227334a FIX: Don’t list values from disabled plugins
Currently, when a plugin registers a new reviewable type or extends a
list method (through `register_reviewble_type` and `extend_list_method`
respectively), the new array is statically computed and always returns
the same value. It will continue to return the same value even if the
plugin is disabled (it can be a problem in a multisite env too).

To address this issue, this patch changes how `extend_list_method`
works. It’s now using `DiscoursePluginRegistry.define_filtered_register`
to create a register on the fly and store the extra values from various
plugins. It then combines the original values with the ones from the
registry. The registry is already aware of disabled plugins, so when a
plugin is disabled, its registered values won’t be returned.
2024-07-11 10:51:48 +02:00
66878a9e80 DEV: Improve logging of Sidekiq errors when logstash logger is enabled (#27855)
This commit improves the logging of Sidekiq errors when
`ENABLE_LOGSTASH_LOGGER` is set to 1. Prior to this change, we would
only log the message and the backtrace. After this change, useful
information like `job.class`, `job.opts`, `job.problem_db`,
`exception.class` and `exception.message` are included in the log line
as well.
2024-07-11 14:17:18 +08:00
ce3d91f422 DEV: Load @json-editor/json-editor using webpack async import (#27773) 2024-07-10 11:15:42 +01:00
758b9dd0ba FEATURE: email attachments in a details (#27804)
This change how we present attachments from incoming emails to now be "hidden" in a "[details]" so they don't "hang" at the end of the post.

This is especially useful when using Discourse as a support tool where email is the main communication channel. For various reasons, images are often duplicated by email user agents, and hiding them behind the details block help keep the conversation focused on the isssue at hand.

Internal ref t/122333
2024-07-10 09:59:27 +02:00
301713ef96 DEV: Upgrade the MessageFormat library (JS)
This patch upgrades the MessageFormat library to version 3.3.0 from
0.1.5.

Our `I18n.messageFormat` method signature is unchanged, and now uses the
new API under the hood.

We don’t need dedicated locale files for handling pluralization rules
anymore as everything is now included by the library itself.

The compilation of the messages now happens through our
`messageformat-wrapper` gem. It then outputs an ES module that includes
all its needed dependencies.

Most of the changes happen in `JsLocaleHelper` and in the `ExtraLocales`
controller.

A new method called `.output_MF` has been introduced in
`JsLocaleHelper`. It handles all the fetching, compiling and
transpiling to generate the proper MF messages in JS. Overrides and
fallbacks are also handled directly in this method.

The other main change is that now the MF translations are served through
the `ExtraLocales` controller instead of being statically compiled in a
JS file, then having to patch the messages using overrides and
fallbacks. Now the MF translations are just another bundle that is
created on the fly and cached by the client.
2024-07-10 09:51:25 +02:00
c9775d5f72 DEV: Apply Logster.store.ignore to DiscourseLogstashLogger as well (#27819)
This commit updates `DiscourseLogstashLogger#add_with_opts` to avoid
logging messages that matches regexp patterns configured in
`Logster.store.ignore`. Those error logs are mostly triggered by clients
and do not serve any useful purpose.
2024-07-10 13:51:42 +08:00
3fee52eae3 PERF: Update docker:test to run QUnit with more cores if available (#27816)
This commit updates the `docker:test` rake task to run core and plugin
QUnit tests in parallel using half the number of available CPU
processors to speed up time it takes to run the tests on hardware with
more CPU cores.

Before this commit, core QUnit tests ran by the `docker:test` rake task was capped at 3 parallel
processes while plugin QUnit tests was not ran in parallel.
2024-07-10 11:27:20 +08:00
b4b7fa17af DEV: Add exception class/message to DiscourseLogstashLogger take 2 (#27815)
This is the second take of af2bd4cc5097501ff0f11f497b8d6f6984cea9ae to
account for messages which contains newlines.
2024-07-10 11:04:17 +08:00
0b64cb9d8c Revert "DEV: Remove git_version from DiscourseLogstashLogger log event (#27730)" (#27814)
This reverts commit bb0daa33cd93eeffb230689d1daacd737f61b2e7.

This commit was not causing the problems we thought it was.
2024-07-10 10:36:22 +08:00
af2bd4cc50 DEV: Add exception class and message fields to DiscourseLogstashLogger (#27787)
This commit updates `DiscourseLogstashlogger` to add the
`exception_class` and `exception_message` field to the log line when the
`progname` of the log message is `web-exception` which is Logster's
logging of exceptions during a web request.

The `exception_class` and `exception_message` fields allows consumers of
the logs to easily group logs together.
2024-07-10 08:54:39 +08:00
560e8aff75 FEATURE: Allow oneboxing private GitHub URLs (#27705)
This commit adds the ability to onebox private GitHub
commits, pull requests, issues, blobs, and actions using
a new `github_onebox_access_token` site setting. The token
must be set up in correctly to have access to the repos needed.

To do this successfully with the Oneboxer, we need to skip
redirects on the github.com host, otherwise we get a 404
on the URL before it is translated into a GitHub API URL
and has the appropriate headers added.
2024-07-10 09:39:31 +10:00
27eea3250c DEV: Fix deprecation typo (#27789) 2024-07-09 17:46:10 +08:00
a461eaddc3 DEV: Refactor ace-editor loading and bump to latest version (#27772)
- Delete vendored copy
- Create a JS entrypoint under `static/` which imports all the modes/themes/extensions we need
- Create an async `load-ace-editor` entrypoint
- Update `<AceEditor` component to use the new entrypoint
- De-jquery-ify `<AceEditor`
- Bump `v1.4.13` -> `v1.35.2`
2024-07-09 10:09:30 +01:00
f44ec18fd2 DEV: update base url links to respect subfolder installs (#27740)
This change eliminates a couple of instances where subfolder urls are badly formatted, in most cases we can use Discourse.base_url_no_prefix to prevent adding the subfolder to the base url.
2024-07-09 12:42:38 +04:00
86e5f46175 DEV: Add hidden s3_inventory_bucket_region site setting (#27786)
This commit adds a hidden `s3_inventory_bucket_region` site setting to
specify the region of the `s3_inventory_bucket` when the `S3Inventory`
class initializes an instance of the `S3Helper`. By default, the
`S3Helper` class uses the value of the `s3_region` site setting but the
region of the `s3_inventory_bucket` is not always the same as the
`s3_region` configured.
2024-07-09 12:03:43 +08:00
17aa831337 DEV: replace imagemagick convert commands with magick (#27767) 2024-07-08 16:55:59 +08:00
28f5550886 DEV: Redo DiscourseLogstashLogger to not rely on logstash-logger (#27759)
This reverts commit 92d7d24d0ffbf3e224af19eda2b24df740df056b.
2024-07-08 14:03:11 +08:00
005f623c42 DEV: Add user_agent column to search_logs (#27742)
Add a new column - `user_agent` - to the `SearchLog` table. 

This column can be null as we are only allowing a the user-agent string to have a max length of 2000 characters. In the case the user-agent string surpasses the max characters allowed, we simply nullify the value, and save/write the log as normal.
2024-07-05 14:05:00 -05:00
92d7d24d0f Revert "DEV: Redo DiscourseLogstashLogger to not rely on logstash-logger (#27663)" (#27733)
This reverts commit 8e10878e1a37a608b0e1705bf9d0a06a295b8a63.

Something is broken on a friday so reverting first before I pick this up
again next Monday.
2024-07-05 17:26:58 +08:00
bb0daa33cd DEV: Remove git_version from DiscourseLogstashLogger log event (#27730)
In our official Docker image, running git commands results in the
following error:

```
fatal: detected dubious ownership in repository at '/var/www/discourse'
To add an exception for this directory, call:

	git config --global --add safe.directory /var/www/discourse
```
2024-07-05 15:22:03 +08:00
964945a11f DEV: Add case-converter common lib and use with AdminPlugin (#27725)
The AdminPlugin JS model uses a similar pattern to chat models,
where it is a plain JS class manually converting provided
snake_case attributes from the serializer to JS camelCase.

However this doesn't work when it comes to using `add_to_serializer`
in plugins since core does not know about these new attributes.

Instead, we can use a JS function to convert snake_case to camelCase
and use that when initializing AdminPlugin. This commit also moves
similar functions to a new case-converter.js file in
discourse-common/lib.
2024-07-05 13:22:48 +10:00
37413c0ecd DEV: Fix unicorn not booting due to 8e10878 (#27727)
`String#present?` is a method defined by Rails on the `String` class but
this class is used before Rails has been loaded.
2024-07-05 10:36:13 +08:00
8e10878e1a DEV: Redo DiscourseLogstashLogger to not rely on logstash-logger (#27663)
This commit rewrites `DiscourseLogstashLogger` to not be an instance
of `LogstashLogger`. The reason we don't want it to be an instance of
`LogstashLogger` is because we want the new logger to be chained to
Logster's logger which can then pass down useful information like the
request's env and error backtraces which Logster has already gathered.

Note that this commit does not bother to maintain backwards
compatibility and drops the `LOGSTASH_URI` and `UNICORN_LOGSTASH_URI`
ENV variables which were previously used to configure the destination in
which `logstash-logger` would send the logs to. Instead, we introduce
the `ENABLE_LOGSTASH_LOGGER` ENV variable to replace both ENV and remove
the need for the log paths to be specified. Note that the previous
feature was considered experimental as stated in d888d3c54c4744d52b9d21d3728f5d6dbc132cde
and the new feature should be considered experimental as well. The code
may be moved into a plugin in the future.
2024-07-05 09:41:52 +08:00
d8e7fc4f5d DEV: check_pending -> check_all_pending! (#27724)
Fixes the following deprecation warning:

> DEPRECATION WARNING: The `check_pending!` method is deprecated in favor of `check_all_pending!`. The new implementation will loop through all available database configurations and find pending migrations. The prior implementation did not permit this.
2024-07-05 09:29:32 +08:00
0dbcc54d4b DEV: use new 'ignore allowed groups' site settings (#27670)
Instead of the deprecated 'min trust level to allow ignore' in order to reduce the number of deprecation notices in the logs.

This tweaks a few serializers so that the 'can_ignore_users?` property is always coming from the server and properly used on the client-side.
2024-07-04 19:27:26 +02:00
8d249457e8 DEV: Upgrade Rails to version 7.1
---------

Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2024-07-04 10:58:21 +02:00
eadda77edf DEV: Drop diffhtml dependency (#27695)
We switched to morphlex via:

- c5f1fc3a08

- bc561eb332
2024-07-04 12:24:51 +10:00
Nat
65be7a7880 Bump version to v3.3.0.beta4-dev 2024-07-03 20:56:17 +08:00
Nat
a29ec30985 Bump version to v3.3.0.beta3 2024-07-03 20:56:17 +08:00
10afe5fcf1 SECURITY: Avoid the use of Object#send in Onebox::Engine::StandardEmbed
Use `Object#public_send` instead which is much safer
2024-07-03 20:49:25 +08:00
311b737c91 SECURITY: Fix Stored-dom XSS via Facebook Oneboxes 2024-07-03 20:49:22 +08:00
67e7808603 SECURITY: Add FastImage SSRF safe freedom patch 2024-07-03 20:49:15 +08:00