Commit Graph

3456 Commits

Author SHA1 Message Date
0eeedf307a DEV: replaces huge generated emoji list by a simpler regex (#11053)
Note that this commit is also fixing various mistakes in emojis.

Some of them have been fixed manually in db.json/data.js/groups.json and will need to be fixed in emoji-db gem.
2021-04-22 08:43:06 +02:00
6b613e3076 FEATURE: Review every post using the review queue. (#12734)
* FEATURE: Review every post using the review queue.

If the `review_every_post` setting is enabled, posts created and edited by regular uses are sent to the review queue so staff can review them. We'll skip PMs and posts created or edited by TL4 or staff users.

Staff can choose  to:

- Approve the post (nothing happens)
- Approve and restore the post (if deleted)
- Approve and unhide the post (if hidden)
- Reject and delete it
- Reject and keep deleted (if deleted)
- Reject and suspend the user
- Reject and silence the user

* Update config/locales/server.en.yml

Co-authored-by: Robin Ward <robin.ward@gmail.com>

Co-authored-by: Robin Ward <robin.ward@gmail.com>
2021-04-21 08:41:36 -03:00
e3b1f5721b FIX: Enforce tag group count validation before sending to review queue (#12728)
There is a category setting that enforces 1 or more tags must be added to a topic from a specific tag group before creating it. This validation was not being run before the topic was being sent to a review queue for categories that have that setting enabled.

There was an existing validation in `TopicCreator` but it was not correct; it was only validating when the tags did _not_ exist and also only happened on `create`. I now run the validation in `TopicCreator.valid?`

I also improved the error message shown to the user when they have not added the tags required (showing the tag names from the tag group), and changed the composer tag selector to not show "optional" if there are N tags required from a certain group.
2021-04-19 09:43:50 +10:00
96a16123d8 FIX: Generate unique HTML heading names (#12705)
Headings with the exact same name generated exactly the same heading
names, which was invalid. This replaces the old code for generating
names for non-English headings which were using URI encode and resulted
in unreadable headings.
2021-04-16 10:54:19 +03:00
0ec5fd5262 DEV: Raise exception when execute_command will spawn a shell (#12716) 2021-04-15 16:29:37 +01:00
1d85574d52 FIX: Don't show the get a room composer message in private categories (#12702) 2021-04-14 12:34:13 -05:00
21d1ee1065 FIX: Use Nokogiri and Loofah consistently (#12693)
CookedPostProcessor used Loofah to parse the cooked content of a post
and Nokogiri to parse cooked Oneboxes. Even though Loofah is built on
top of Nokogiri, replacing an element from the cooked post (a Nokogiri
node) with a parsed onebox (a Loofah node) produced a strange result
which included XML namespaces. Removing the mix and using Loofah
to parse Oneboxes fixed the problem.
2021-04-14 18:09:55 +03:00
e4e2c7c66f FIX: Improve anchor links (#12683)
* FIX: Use theme color for anchor icon

* FIX: Do not count anchor links

* FIX: Do not count hashtags links either

* DEV: Add tests for link_count

* FIX: Disable anchors in quotes and preview

* FIX: Try building some anchor slugs for unicode

* DEV: Fix tests
2021-04-14 10:27:07 +03:00
eeaecd4fd2 FEATURE: Category setting to allow unlimited first post edits by the owner of the topic (#12690)
This PR adds a new category setting which is a column in the `categories` table, `allow_unlimited_owner_edits_on_first_post`.

What this does is:

* Inside the `can_edit_post?` method of `PostGuardian`, if the current user editing a post is the owner of the post, it is the first post, and the topic's category has `allow_unlimited_owner_edits_on_first_post`, then we bypass the check for `LimitedEdit#edit_time_limit_expired?` on that post.
* Also, similar to wiki topics, in `PostActionNotifier#after_create_post_revision` we send a notification to all users watching a topic when the OP is edited in a topic with the category setting `allow_unlimited_owner_edits_on_first_post` enabled.

This is useful for forums where there is a Marketplace or similar category, where topics are created and then updated indefinitely by the OP rather than the OP making new topics or additional replies. In a way this acts similar to a wiki that only one person can edit.
2021-04-14 15:54:09 +10:00
65647000a1 DEV: Allow Discourse::Utils.execute_command timeout with env (#12672)
Followup to 5deda5ef3ef8763dfe936a313e0dddaa279a1b04

The first argument to `Open3.capture3` can be an environment variable hash. In this case, we need to insert the `timeout` command after the env hash.
2021-04-12 13:53:41 +01:00
abb0a4bae2 DEV: Add SCSS helper to replace asset-uri and image-uri (#12664) 2021-04-12 13:57:39 +10:00
Sam
5deda5ef3e FIX: automatically timeout long running image magick commands (#12670)
Previously certain images may lead to convert / identify to run for unreasonable
amounts of time

This adds a maximum amount of time these commands can run prior to forcing
them to stop
2021-04-12 13:55:54 +10:00
Sam
5b342ae505 FIX: remove superfluous spaces from CJK blurbs (#12629)
Previously we used the raw data indexed to generate blurbs even for cases
when Chinese/Korean/Japanese text was used.

This caused superfluous spaces to show up in excerpts.
2021-04-12 12:46:42 +10:00
50252d803e DEV: stub youtube embed requests (#12637)
* DEV: stub youtube embed requests

* DEV: Ignore redirects on youtube.com when oneboxing
2021-04-07 13:32:27 -04:00
5e4c0e2caa FEATURE: Treat site settings as plain text and add a new HTML type. (#12618)
To add an extra layer of security, we sanitize settings before shipping them to the client. We don't sanitize those that have the "html" type.

The CookedPostProcessor already uses Loofah for sanitization, so I chose to also use it for this. I added it to our gemfile since we installed it as a transitive dependency.
2021-04-07 12:51:19 -03:00
2308a58113 DEV: Added support for custom site setting 'emoji_list' (#12414)
Example usage:

```
best_emojis:
    type: emoji_list
    default: laughing|open_mouth|cry|angry|hugs
    client: true
```
2021-04-07 15:32:05 +02:00
e026af11ac FIX: Simplify send PM to email settings (#12583)
This feature used to be controlled by two site settings 
enable_personal_email_messages and min_trust_to_send_email_messages.
I removed enable_personal_email_messages and unhide 
min_trust_to_send_email_messages to simplify the process of 
enabling / disabling this feature.
2021-04-06 15:39:27 +03:00
6234d7455b FEATURE: add maximum limit for secondary emails (#12599) 2021-04-05 20:31:42 +05:30
c478ffc662 FIX: post merging was failing silently (#12566)
https://meta.discourse.org/t/merging-very-long-posts-removes-them/183597
2021-04-01 06:46:18 +05:30
68d0916eb5 FEATURE: Oneboxer cache response body (#12562)
* FEATURE: Cache successful HTTP GET requests during Oneboxing

Some oneboxes may fail if when making excessive and/or odd requests against the target domains. This change provides a simple mechanism to cache the results of succesful GET requests as part of the oneboxing process, with the goal of reducing repeated requests and ultimately improving the rate of successful oneboxing.

To enable:

Set `SiteSetting.cache_onebox_response_body` to `true`

Add the domains you’re interesting in caching to `SiteSetting. cache_onebox_response_body_domains` e.g. `example.com|example.org|example.net`

Optionally set `SiteSetting.cache_onebox_user_agent` to a user agent string of your choice to use when making requests against domains in the above list.

* FIX: Swap order of duration and value in redis call

The correct order for `setex` arguments is `key`, `duration`, and `value`.

Duration and value had been flipped, however the code would not have thrown an error because we were caching the value of `1.day.to_i` for a period of 1 seconds… The intention appears to be to set a value of 1 (purely as a flag) for a period of 1 day.
2021-03-31 13:19:34 -04:00
52a5c7e438 FIX: Improve error messages if user cannot send PM emails (#12547) 2021-03-30 12:18:57 +03:00
8de07181a8 FEATURE: Add new :topic_tags_changed DiscourseEvent (#12530)
This is called in DiscourseTagging.tag_topic_by_names only after
all the validations etc. have been passed, and after topic.tags = X
has been called (because this is when the associations are created/
destroyed). The event has the topic, then a second param with the
old and new tag names in arrays for easy inspection.
2021-03-26 13:53:47 +10:00
c03c85e661 FIX: delete orphan post revisions (#12502)
I was adding specs to ensure that post actions and uploads are removed for permanently deleted posts.

I noticed that post revisions were not permanently destroyed. I added a migration to fix old data.
2021-03-25 12:34:53 +11:00
ea6f9af08b FIX: Add topic_diff to PostRevisor (#12518)
The instance of the PostRevisor is passed to the post_edited
event. It is useful to know what has happened to the topic in
this event (we already pass a boolean for topic_changed? but that
is not so helpful by itself).
2021-03-25 10:24:50 +10:00
e4f3a04d53 DEV: Move color definition functions to mixins (#12511) 2021-03-24 18:35:52 -04:00
e7fb45cc29 FEATURE: Treat emoji or similar characters as one (#12482)
Long messages consisting only of emojis, dots or commas used to be
valid because character-wise they were over the limit.
2021-03-24 16:47:35 +02:00
6ff888bd2c DEV: Retry-after header values should be strings (#12475)
Fixes `Rack::Lint::LintError: a header value must be a String, but the value of 'Retry-After' is a Integer`. (see: 14a236b4f0/lib/rack/lint.rb (L676))

I found it when I got flooded by those warning a while back in a test-related accident 😉 (ember CLI tests were hitting a local rails server at a fast rate)
2021-03-23 20:32:36 +01:00
066c59d0e3 FIX: Regression in colors used by non-default theme (#12492)
eb7f0ec caused this regression, where a non-default theme set to use
the base color scheme was resolving to the default theme's color scheme.
2021-03-23 12:46:25 -04:00
2ad9b3f432 FEATURE: Add anchor links to headings (#12379) 2021-03-23 10:45:06 +02:00
4e46732346 FEATURE: Implement browser update in crawler view (#12448)
browser-update script does not work correctly in some very old browsers
because the contents of <noscript> is not accessible in JavaScript.
For these browsers, the server can display the crawler page and add the
browser update notice.

Simply loading the browser-update script in the crawler view is not a
solution because that means all crawlers will also see it.
2021-03-22 19:41:42 +02:00
6eb0d0c38d SECURITY: Fix is_private_ip for RateLimiter to cover all cases (#12464)
The regular expression to detect private IP addresses did not always detect them successfully.
Changed to use ruby's in-built IPAddr.new(ip_address).private? method instead
which does the same thing but covers all cases.
2021-03-22 13:56:32 +10:00
355d51afde FEATURE: Allow using invites when DiscourseConnect SSO is enabled (#12419)
This PR allows invitations to be used when the DiscourseConnect SSO is enabled for a site (`enable_discourse_connect`) and local logins are disabled. Previously invites could not be accepted with SSO enabled simply because we did not have the code paths to handle that logic.

The invitation methods that are supported include:

* Inviting people to groups via email address
* Inviting people to topics via email address
* Using invitation links generated by the Invite Users UI in the /my/invited/pending route

The flow works like this:

1. User visits an invite URL
2. The normal invitation validations (redemptions/expiry) happen at that point
3. We store the invite key in a secure session
4. The user clicks "Accept Invitation and Continue" (see below)
5. The user is redirected to /session/sso then to the SSO provider URL then back to /session/sso_login
6. We retrieve the invite based on the invite key in secure session. We revalidate the invitation. We show an error to the user if it is not valid. An additional check here for invites with an email specified is to check the SSO email matches the invite email
7. If the invite is OK we create the user via the normal SSO methods
8. We redeem the invite and activate the user. We clear the invite key in secure session.
9. If the invite had a topic we redirect the user there, otherwise we redirect to /

Note that we decided for SSO-based invites the `must_approve_users` site setting is ignored, because the invite is a form of pre-approval, and because regular non-staff users cannot send out email invites or generally invite to the forum in this case.

Also deletes some group invite checks as per https://github.com/discourse/discourse/pull/12353
2021-03-19 10:20:10 +10:00
033d6b6437 FEATURE: Obfuscate emails on invite show page (#12433)
The email should not be ever displayed in clear text, except the case
when the user authenticates using another service.
2021-03-18 19:09:23 +02:00
da210b6d77 FEATURE: Replace markdown-it replacements rule. (#12417)
We override the default replacements rule to no longer replace "(c)", "(p)", and "(p)". Additionally, we merged the custom arrows rule into the replacement function.
2021-03-18 10:55:41 -03:00
eb7f0ec766 FIX: In components, fall back to default theme color variables (#12423)
Component SCSS compilation should use the current theme's SCSS color
variables as a fallback before using the default core colors.

This is mostly a backwards-compatibility fix, new themes and components
should use CSS custom properties, which offer better support for on-the-fly
color scheme changes (dark mode support, etc.).
2021-03-17 13:34:15 -04:00
Sam
6c57f6f49d DEV: ensure state is clean prior to spec (#12399)
Corrects flaky spec

Previously we were only clearing state after our spec ran, leaving possible
edge cases where `Discourse.plugins` had data.

Clean-up source of the plugin leak

1 pop was not enough to clear the plugin, plus make specs a bit more
deliberate
2021-03-15 09:11:23 +01:00
52d833472c DEV: Refactor plugin SCSS compilation (#12359) 2021-03-12 11:17:42 -05:00
56262349a9 FIX: When destroying a topic, destroy its associated published page. (#12358)
Destroying a topic should remove the associated published page so another topic could re-use the slug.
2021-03-11 11:34:54 -03:00
8fcad73b36 FEATURE: Admins can flag posts so they can review them later. (#12311)
Staff can send a post to the review queue by clicking the "Flag Post" button next to "Take Action...". Clicking it flags the post using the "Notify moderators" score type and hides it. A custom message will be sent to the user.
2021-03-11 08:21:24 -03:00
10780d2448 DEV: support json_schema in theme settings (#12294) 2021-03-10 20:15:04 -05:00
Sam
9fb9a2c098 DEV: freeze time when running rate limiter tests (#12315)
This avoids issues around clock skew making retry-after return 9 instead of
10
2021-03-11 10:47:23 +11:00
aed97c7bab FIX: Add amazon sites to force_get_hosts (#12341)
It has been observed that doing a HEAD against an Amazon store URL may result in a 405 error being returned.

Skipping the HEAD request may result in an improved oneboxing experience when requesting these URLs.
2021-03-10 14:42:17 -05:00
4071af1d09 DEV: Refactor font and category background importers (#12312) 2021-03-10 11:05:56 -05:00
9bd436c20b FIX: Do not add same email multiple times (#12322)
The user and an admin could create multiple email change requests for
the same user. If any of the requests was validated and it became
primary, the other request could not be deleted anymore.
2021-03-10 14:49:26 +02:00
ce04db8610 FEATURE: Allow invites redemption with Omniauth providers. 2021-03-09 09:27:18 +08:00
ebe4896e48 FEATURE: Change very high/low search priority to rank at absolute ends.
Prior to this change, we had weights for very_high, high, low and
very_low. This means there were 4 weights to tweak and what weights to
use for `very_high/high` and `very_low/low` pair was hard to explain.
This change makes it such that `very_high` search priority will always
ensure that the posts are ranked at the top while `very_low` search
priority will ensure that the posts are ranked at the very bottom.
2021-03-09 09:20:37 +08:00
38ab45cb93 FIX: check min tag count requirement when change category (#12252)
When a category is created, we can set `minimum_required_tags` property.

When the topic is created, we are checking that field and ensuring that the minimum amount of tags were added - unless topic created by a staff member.

Problem is that validation is skipped when we change the category from for example category with 0 tags required to the category with 1 tag required.

Changing category is kind of the unicorn as it is a complicated operation:

https://github.com/discourse/discourse/blob/master/lib/post_revisor.rb#L84

https://github.com/discourse/discourse/blob/master/app/models/topic.rb#L911

https://github.com/discourse/discourse/blob/master/app/models/topic.rb#L823

Before we start to try to change the category, we should ensure that the tags requirement is fulfilled.

https://meta.discourse.org/t/the-category-setting-for-tags-is-not-respected/181214
2021-03-03 10:59:23 +11:00
83f332b5a5 FEATURE: Add a site setting to allow emojis to come from an external URL (#12180) 2021-03-02 16:04:16 -03:00
8fd46c04ea Drop flash video onebox (#12261)
Flash was discontinued by Adobe at the end of 2020. There is no need to continue OneBox support for it
2021-03-02 17:11:14 +00:00
c0e2fdd200 FIX: Components mobile-specific CSS was missing (#12259)
Fix for: https://meta.discourse.org/t/our-components-stop-working/181580?u=osama.

This fixes an old hidden bug that was exposed in cf0192018e. The bug is that we call the `Stylesheet::Manager.stylesheet_details` method with the `target` arg as `:mobile_theme` when we want to retrieve a theme component's mobile CSS. The problem is that this `target` value will at some point be looked up in the `Theme.targets` enum which doesn't have a `:mobile_theme` key, instead it has `:mobile` key.

This commit adds a step that removes the `_theme` suffix in the `Theme.list_baked_fields` method to fix this problem.
2021-03-02 17:20:43 +03:00