Commit Graph

12475 Commits

Author SHA1 Message Date
25778d9861 FIX: Return 400 response codes when topic list query params are invalid (#27930)
This commit updates `TopicQuery.validators` to cover all of the
public options listed in `TopicQuery.public_valid_options`. This is done
to fix the app returning a 500 response code when an invalid value, such
as a hash, is passed as a query param when accessing the various topic
list routes.
2024-07-16 10:30:04 +08:00
00608a19c6 FIX: Show the SMTP authentication error for group UI (#27914)
Originally in 964da218173db007fefe6357e96292f5545c513e
we hid the SMTPAuthenticationError message except in
very specific cases. However this message often contains
helpful information from the mail provider, for example
here is a response from Office365:

> 535 5.7.139 Authentication unsuccessful, user is locked by your
organization's security defaults policy. Contact your administrator.

So, we will show the error message in the modal UI instead
of supressing it with a generic message to be more helpful.
2024-07-16 09:14:17 +10:00
7b53e610c1 SECURITY: limit the number of characters in watched word replacements.
The watch words controller creation function, create_or_update_word(), doesn’t validate the size of the replacement parameter, unlike the word parameter, when creating a replace watched word. So anyone with moderator privileges can create watched words with almost unlimited characters.
2024-07-15 19:25:17 +08:00
a3d319ac2f FIX: StaticController#enter should not redirect to invalid paths (#27913)
This commit updates `StaticController#enter` to not redirect to invalid
paths when the `redirect` param is set. Instead it should redirect to `/` when the
`redirect` param is invalid.
2024-07-15 14:39:37 +08:00
97e2b353f6 FEATURE: Allow for multiple GitHub onebox tokens (#27887)
Followup 560e8aff75e4bde67bb162e8fdea52e704a19f81

GitHub auth tokens cannot be made with permissions to
access multiple organisations. This is quite limiting.
This commit changes the site setting to be a "secret list"
type, which allows for a key/value mapping where the value
is treated like a password in the UI.

Now when a GitHub URL is requested for oneboxing, the
org name from the URL is used to determine which token
to use for the request.

Just in case anyone used the old site setting already,
there is a migration to create a `default` entry
with that token in the new list setting, and for
a period of time we will consider that token valid to
use for all GitHub oneboxes as well.
2024-07-15 13:07:36 +10:00
06131bd4fd FIX: Don't require fields required on sign-up when updating fields (#27888)
### What is the problem?

We have recently added a new option to add user fields required for existing users. This is in contrast to requiring fields only on sign-up.

This revealed an existing problem. Consider the following:

1. User A signs up.
2. Admin adds a new user field required on sign-up. (Should not apply to User A since they already signed up.)
3. User A tries to update their profile.

**Expected behaviour:**

No problem.

**Actual behaviour:**

User A receives an error saying they didn't fill up all required fields.

### How does this fix it?

When updating profile, we only check that required fields that are "for all users" are filled. Additionally, we check that fields that were required on sign-up and have previously been filled are not blanked out.
2024-07-15 09:56:20 +10:00
9e4e591d60 Revert "FEATURE: custom flag can require additional message (#27706)" (#27906)
This reverts commit c0bcd979e38134989b9023b75738caa743d2e092.
2024-07-15 09:45:57 +10:00
367040024b DEV: unparallel flags system spec (#27903)
DEV: unparallel flags system spec

Because of the global flag cache, those specs should not run in parallel. We need to execute them sequentially.
2024-07-15 07:36:54 +08:00
c0bcd979e3 FEATURE: custom flag can require additional message (#27706)
Allow admin to create custom flag which requires an additional message.

I decided to rename the old `custom_flag` into `require_message` as it is more descriptive.
2024-07-15 08:48:01 +10:00
63ca30ccb4 FIX: Don't let table-build automatically fill empty headers with default values (#27894)
* FIX: Don't let table-build automatically fill empty headers with default values

The old table builder would fill empty headers with default values A~Z when editing.
This commit makes table-builder respect the old empty headers

related meta topic: https://meta.discourse.org/t/editing-a-table-with-empty-headers-fills-them-in-with-the-default-text-column-a-column-b/268472
2024-07-13 00:41:18 +08:00
7f0e6e9592 FIX: Allow error handling for formats besides JSON (#27811)
* Allow error handling for formats besides JSON

* Add a test and sets the default format as JSON
2024-07-11 11:59:00 -03:00
3978db0811 DEV: Add missing Chinese simplified to names.yml (#27847) 2024-07-11 07:54:45 -07:00
b0480dd34e DEV: Avoid instance variables in specs
Small followup of https://github.com/discourse/discourse/pull/27705
2024-07-11 14:31:20 +02:00
5ec227334a FIX: Don’t list values from disabled plugins
Currently, when a plugin registers a new reviewable type or extends a
list method (through `register_reviewble_type` and `extend_list_method`
respectively), the new array is statically computed and always returns
the same value. It will continue to return the same value even if the
plugin is disabled (it can be a problem in a multisite env too).

To address this issue, this patch changes how `extend_list_method`
works. It’s now using `DiscoursePluginRegistry.define_filtered_register`
to create a register on the fly and store the extra values from various
plugins. It then combines the original values with the ones from the
registry. The registry is already aware of disabled plugins, so when a
plugin is disabled, its registered values won’t be returned.
2024-07-11 10:51:48 +02:00
66878a9e80 DEV: Improve logging of Sidekiq errors when logstash logger is enabled (#27855)
This commit improves the logging of Sidekiq errors when
`ENABLE_LOGSTASH_LOGGER` is set to 1. Prior to this change, we would
only log the message and the backtrace. After this change, useful
information like `job.class`, `job.opts`, `job.problem_db`,
`exception.class` and `exception.message` are included in the log line
as well.
2024-07-11 14:17:18 +08:00
7b627dc14b FIX: Office365/Outlook auth method for group SMTP (#27854)
Both office365 and outlook SMTP servers need LOGIN
SMTP authentication instead of PLAIN (which is what
we are using by default). This commit uses that
unconditionally for these servers, and also makes
sure to use STARTTLS for them too.
2024-07-11 16:16:54 +10:00
0e3ed7ea2a A11Y: improve topic list table markup for screenreaders (#27808)
Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2024-07-10 13:14:36 -04:00
ab99f31760 DEV: Fix the I18n integrity spec
Before Rails 7.1, the `config.i18n.raise_on_missing_translations` option
was raising only in controllers and views, now it’s anywhere in the app.
It means it raises each time `#description` is called for a setting that
is missing a proper description (and we have a ton of them). Most of the
time it’s fine, as those are usually settings that aren’t shown to the
user.

We can’t just let the code blow up every time there’s a setting with a
missing description, that’s why it’s currently returning an empty
string when the translation is missing.

However, this silently broke our I18n integrity spec that was relying on
the old “Translation missing” message to detect missing translations.

This patch addresses this issue by checking the description isn’t an
empty string. It caught a missing translation by the way.
2024-07-10 11:39:13 +02:00
758b9dd0ba FEATURE: email attachments in a details (#27804)
This change how we present attachments from incoming emails to now be "hidden" in a "[details]" so they don't "hang" at the end of the post.

This is especially useful when using Discourse as a support tool where email is the main communication channel. For various reasons, images are often duplicated by email user agents, and hiding them behind the details block help keep the conversation focused on the isssue at hand.

Internal ref t/122333
2024-07-10 09:59:27 +02:00
301713ef96 DEV: Upgrade the MessageFormat library (JS)
This patch upgrades the MessageFormat library to version 3.3.0 from
0.1.5.

Our `I18n.messageFormat` method signature is unchanged, and now uses the
new API under the hood.

We don’t need dedicated locale files for handling pluralization rules
anymore as everything is now included by the library itself.

The compilation of the messages now happens through our
`messageformat-wrapper` gem. It then outputs an ES module that includes
all its needed dependencies.

Most of the changes happen in `JsLocaleHelper` and in the `ExtraLocales`
controller.

A new method called `.output_MF` has been introduced in
`JsLocaleHelper`. It handles all the fetching, compiling and
transpiling to generate the proper MF messages in JS. Overrides and
fallbacks are also handled directly in this method.

The other main change is that now the MF translations are served through
the `ExtraLocales` controller instead of being statically compiled in a
JS file, then having to patch the messages using overrides and
fallbacks. Now the MF translations are just another bundle that is
created on the fly and cached by the client.
2024-07-10 09:51:25 +02:00
6591a0654b FIX: Destroy Drafts when increasing sequences (#27739)
Drafts used to be deleted instead of being destroyed. The callbacks that
clean up the upload references were not being called. As a result, the
upload references were not cleaned up and uploads were not deleted
either. This has been partially fixed in 9655bf3e.
2024-07-10 10:43:11 +03:00
c9775d5f72 DEV: Apply Logster.store.ignore to DiscourseLogstashLogger as well (#27819)
This commit updates `DiscourseLogstashLogger#add_with_opts` to avoid
logging messages that matches regexp patterns configured in
`Logster.store.ignore`. Those error logs are mostly triggered by clients
and do not serve any useful purpose.
2024-07-10 13:51:42 +08:00
b4b7fa17af DEV: Add exception class/message to DiscourseLogstashLogger take 2 (#27815)
This is the second take of af2bd4cc5097501ff0f11f497b8d6f6984cea9ae to
account for messages which contains newlines.
2024-07-10 11:04:17 +08:00
0b64cb9d8c Revert "DEV: Remove git_version from DiscourseLogstashLogger log event (#27730)" (#27814)
This reverts commit bb0daa33cd93eeffb230689d1daacd737f61b2e7.

This commit was not causing the problems we thought it was.
2024-07-10 10:36:22 +08:00
af2bd4cc50 DEV: Add exception class and message fields to DiscourseLogstashLogger (#27787)
This commit updates `DiscourseLogstashlogger` to add the
`exception_class` and `exception_message` field to the log line when the
`progname` of the log message is `web-exception` which is Logster's
logging of exceptions during a web request.

The `exception_class` and `exception_message` fields allows consumers of
the logs to easily group logs together.
2024-07-10 08:54:39 +08:00
560e8aff75 FEATURE: Allow oneboxing private GitHub URLs (#27705)
This commit adds the ability to onebox private GitHub
commits, pull requests, issues, blobs, and actions using
a new `github_onebox_access_token` site setting. The token
must be set up in correctly to have access to the repos needed.

To do this successfully with the Oneboxer, we need to skip
redirects on the github.com host, otherwise we get a 404
on the URL before it is translated into a GitHub API URL
and has the appropriate headers added.
2024-07-10 09:39:31 +10:00
bbd67eff08 DEV: Improve the sidebar section expansion handling (#27805)
Handles the cases where the sections titles are Unicode only strings, allowing them to be expanded separately if the Unicode string contains letters.

Also prevents a sidebar section with the header hidden to be displayed collapsed.
2024-07-09 18:32:29 -03:00
7a7bdc9be5 FEATURE: Use group based setting for unsafe-none COOP (#27783)
Followup 3ff7ce78e782c7d28c8b5a1a3f40a1de897d89a1

Basing this setting on referrer was too brittle --
the referrer header can easily be ommitted or changed.
Instead, for the small amount of use cases that this
site setting serves, we can use a group-based setting
instead, changing it to `cross_origin_opener_unsafe_none_groups`
instead.
2024-07-09 11:25:49 -05:00
a01be4150a DEV: Specs for redeliver_web_hook_events job (#27779)
It is a PR to add a spec for checking redeliver_web_hook_events job not to delete webhook event in process.
2024-07-09 10:35:10 -05:00
0846862cb5 FIX: deleted topic author in crawler view (#27788)
When a crawler visits a topic that has a deleted author, it would error because the `show.html.erb` view was expecting a user to be always present.

This ensure we don't render the "author" meta data when the author of the topic has been deleted.

Internal ref t/132508
2024-07-09 10:44:03 +02:00
f44ec18fd2 DEV: update base url links to respect subfolder installs (#27740)
This change eliminates a couple of instances where subfolder urls are badly formatted, in most cases we can use Discourse.base_url_no_prefix to prevent adding the subfolder to the base url.
2024-07-09 12:42:38 +04:00
e58cf24fcc FEATURE: Topic view stats report (#27760)
Adds a report to show the top 100 most viewed topics in a date range,
combining logged in and anonymous views. Can be filtered by category.

This is a followup to 527f02e99fee782f53e206f739fb4f12d63b6d2a
and d1191b7f5fbe9e8ea1b645d5a61c79fa93693c0c. We are also going to
be able to see this data in a new topic map, but this admin report
helps to see an overview across the forum for a date range.
2024-07-09 15:39:10 +10:00
86e5f46175 DEV: Add hidden s3_inventory_bucket_region site setting (#27786)
This commit adds a hidden `s3_inventory_bucket_region` site setting to
specify the region of the `s3_inventory_bucket` when the `S3Inventory`
class initializes an instance of the `S3Helper`. By default, the
`S3Helper` class uses the value of the `s3_region` site setting but the
region of the `s3_inventory_bucket` is not always the same as the
`s3_region` configured.
2024-07-09 12:03:43 +08:00
7111d5e4bf DEV: Fix flaky admin confirmation spec (#27784)
Waiting for the dialog to close was not enough,
need to wait for the overridden indicator to
show on the site setting.
2024-07-09 12:05:31 +10:00
784c04ea81 FEATURE: Add Mechanism to redeliver all failed webhook events (#27609)
Background:
In order to redrive failed webhook events, an operator has to go through and click on each. This PR is adding a mechanism to retry all failed events to help resolve issues quickly once the underlying failure has been resolved.

What is the change?:
Previously, we had to redeliver each webhook event. This merge is adding a 'Redeliver Failed' button next to the webhook event filter to redeliver all failed events. If there is no failed webhook events to redeliver, 'Redeliver Failed' gets disabled. If you click it, a window pops up to confirm the operator. Failed webhook events will be added to the queue and webhook event list will show the redelivering progress. Every minute, a job will be ran to go through 20 events to redeliver. Every hour, a job will cleanup the redelivering events which have been stored more than 8 hours.
2024-07-08 15:43:16 -05:00
17aa831337 DEV: replace imagemagick convert commands with magick (#27767) 2024-07-08 16:55:59 +08:00
28f5550886 DEV: Redo DiscourseLogstashLogger to not rely on logstash-logger (#27759)
This reverts commit 92d7d24d0ffbf3e224af19eda2b24df740df056b.
2024-07-08 14:03:11 +08:00
c3598847fe DEV: Truncate user agent string when it is too long instead of null (#27758)
This is a follow up to 005f623c42b0e18392cc0ad29432961b3df7dccb where
we want to truncate the user agent string instead of nulling out the
column when the user agent string is too low. By truncating, we still
get to retain information that can still be useful.
2024-07-08 13:58:20 +08:00
df6f950200 DEV: Hide admin Moderation Flags UI behind feature flag for now (#27756)
Adds experimental_flags_admin_page_enabled_groups (default "")
to remove the Moderation Flags link from the admin sidebar for now,
there are still a few bugfixes that need to be done before we
are comfortable with turning this on more widely. This is
a _temporary_ flag, we will be removing this once the feature
is more stable.
2024-07-08 11:09:30 +10:00
6022cc2af8 DEV: Escape the sidebar filter and admin sidebar no results description (#27746) 2024-07-05 17:54:22 -03:00
005f623c42 DEV: Add user_agent column to search_logs (#27742)
Add a new column - `user_agent` - to the `SearchLog` table. 

This column can be null as we are only allowing a the user-agent string to have a max length of 2000 characters. In the case the user-agent string surpasses the max characters allowed, we simply nullify the value, and save/write the log as normal.
2024-07-05 14:05:00 -05:00
b36cbc7d21 DEV: Untangle the admin sidebar from the sidebar code (#27640) 2024-07-05 13:11:15 -03:00
640dccd224 FIX: show primary user group options to members in account prefs (#27664)
The user serializer groups method previously relied on the members_visible_groups to determine groups that the user should be able to see, however this setting was intended for visibility of group members (which is entirely different).

The result of this could be seen when choosing a primary group from user preferences -> account, due to the serializer the group name was not visible when members_visible_groups was set to owners.
2024-07-05 19:43:50 +04:00
1bc18a5bf4 DEV: Remove spec following revert in 92d7d24d0f (#27736)
This spec was introduced as part of the original commit (8e10878e1a), but was missed from the revert.
2024-07-05 12:28:16 +01:00
1045d22b41 FIX: show cooked bio in user profile (#27734)
When editing your bio in your user profile and then saving it, it would be removed from the profile header section.

This was broken for over 7 years since ae68795d8206a6a796d7d65e63bd2af3509be704 was merged 😅

Context - https://meta.discourse.org/t/after-saving-changes-to-profile-expanding-full-profile-only-shows-half/314118
2024-07-05 12:31:57 +02:00
bb0daa33cd DEV: Remove git_version from DiscourseLogstashLogger log event (#27730)
In our official Docker image, running git commands results in the
following error:

```
fatal: detected dubious ownership in repository at '/var/www/discourse'
To add an exception for this directory, call:

	git config --global --add safe.directory /var/www/discourse
```
2024-07-05 15:22:03 +08:00
72fd509fd4 DEV: Comment flaky specs (#27729)
Followup 527f02e99fee782f53e206f739fb4f12d63b6d2a,
don't have time to investigate now, will look next week
2024-07-05 14:45:54 +10:00
33c68b28b6 DEV: Use serializers for user_notification_schedule and featured_topic (#27719)
Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2024-07-05 00:00:24 -03:00
8e10878e1a DEV: Redo DiscourseLogstashLogger to not rely on logstash-logger (#27663)
This commit rewrites `DiscourseLogstashLogger` to not be an instance
of `LogstashLogger`. The reason we don't want it to be an instance of
`LogstashLogger` is because we want the new logger to be chained to
Logster's logger which can then pass down useful information like the
request's env and error backtraces which Logster has already gathered.

Note that this commit does not bother to maintain backwards
compatibility and drops the `LOGSTASH_URI` and `UNICORN_LOGSTASH_URI`
ENV variables which were previously used to configure the destination in
which `logstash-logger` would send the logs to. Instead, we introduce
the `ENABLE_LOGSTASH_LOGGER` ENV variable to replace both ENV and remove
the need for the log paths to be specified. Note that the previous
feature was considered experimental as stated in d888d3c54c4744d52b9d21d3728f5d6dbc132cde
and the new feature should be considered experimental as well. The code
may be moved into a plugin in the future.
2024-07-05 09:41:52 +08:00
0dbcc54d4b DEV: use new 'ignore allowed groups' site settings (#27670)
Instead of the deprecated 'min trust level to allow ignore' in order to reduce the number of deprecation notices in the logs.

This tweaks a few serializers so that the 'can_ignore_users?` property is always coming from the server and properly used on the client-side.
2024-07-04 19:27:26 +02:00