Commit Graph

7593 Commits

Author SHA1 Message Date
80e17f5f9e DEV: Remove staff bypass on fix missing
After thinking about it, I worry that this will potentially leave a site
setting set when people hit ctrl-c ... feels a tiny bit risky, so leaving
it out.
2020-08-28 12:35:35 +10:00
ed9323043f DEV: Allow all uploads when fixing missing s3
If we do not do this we can not properly re-download files
2020-08-28 12:28:59 +10:00
2352f4bfc7 DEV: Replace SECURE_MEDIA_ROUTE const with other methods (#10545)
This is so if the route changes this const won't be around to bite us, use the Rails route methods instead.
2020-08-28 11:28:11 +10:00
a3577435f7 FEATURE: Additional control of iframes in oneboxes (#10523)
This commit adds a new site setting "allowed_onebox_iframes". By default, all onebox iframes are allowed. When the list of domains is restricted, Onebox will automatically skip engines which require those domains, and use a fallback engine.
2020-08-27 20:12:13 +01:00
c0cd69d280 DEV: Restoring backups didn't work on macOS
This is a follow-up for f51ccea02816c768863b1073977db219f4b66ff3 because specs were failing on macOS. macOS uses bsdtar by default and the --transform option isn't supported by bsdtar. But the -s option is and it works the same.
2020-08-27 19:08:47 +02:00
188d224303 DEV: Restore UploadRecovery specs (#10543)
Originally disabled in 0c0192e. Upload specs now use separate paths for each spec worker.

Fixes an issue in UploadRecovery#recover_from_local – it didn't take into account the testing infix (e.g. test_0) in the uploads/tombstone paths.
2020-08-27 15:57:10 +02:00
40c6d90df3 PERF: Create a partial regular post_search_data index on large sites.
With the addition of `PostSearchData#private_message`, a partial
index consisting of only search data from regular posts can be created.
The partial index helps to speed up searches on large sites since PG
will not have to do an index scan on the entire search data index which
has shown to be a bottle neck.
2020-08-27 13:42:00 +08:00
f7314f8ab4 DEV: Improve s3 upload image analysis
- Introduces uploads:delete_missing_s3 which can be used to "give up" and
delete broken records from the database

- Fixes a bug in fix_missing_s3 - crashing on deleted posts

- Adds more info to analyze_missing_s3
2020-08-27 11:50:07 +10:00
618a7ecb35 FIX: default_tags_muted site setting won't have tag ids.
Instead it only have list of tag names separated by comma.
89fcb75af2b8abfd146ab9b70895db466c0d2a9d
2020-08-26 23:05:29 +05:30
b60c39fdf0 DEV: search more carefully for missing uploads
rake uploads:analyze_missing_s3 was not looking at all places, amend it so
it looks in all the places where uploads could exist.
2020-08-26 17:48:42 +10:00
2523a21fc3 FIX: Server side rendering was broken with new helperContext() 2020-08-25 09:16:32 -04:00
1a2800ad07 fix: 🐛 category & tag search regex to support thai character 2020-08-25 16:12:26 +08:00
629ee5494d FEATURE: Allow plugins to register parameter-based API routes (#10505)
Example usage:

```
add_api_parameter_route(
  method: :get,
  route: "users#bookmarks",
  format: :ics
)
```
2020-08-24 10:24:52 +01:00
e8a842ab8c FIX: Stop category logo + background being marked secure (#10513)
Meta topic: https://meta.discourse.org/t/secure-media-uploads-breaks-category-logos/161693

Category backgrounds and logos are public uploads and should not be marked as secure.

I also discovered that a lot of the UploadSecurity specs for public types were returning false positives; this has been fixed.
2020-08-24 17:12:28 +10:00
05174df5c0 FIX: Restrict personal_messages: advanced search filter to admin.
The filter noops if an incorrect username is passed. This filter is not
exposed as part of the UI but is only used when an admin transitions
from a search within a user's personal messages to the full page search.

Follow-up to 4b3079905498e3d09517ee2766c8ff33c11e7ada.
2020-08-24 13:53:48 +08:00
c6ceda8c4e PERF: Avoid extra subquery when searching within PMs for normal user.
Note the following query being generated where the filter for a user's
private messages is executed twice.

```sql
SELECT "posts"."id", "posts"."user_id", "posts"."topic_id", "posts"."post_number", "posts"."raw", "posts"."cooked", "posts"."created_at", "posts"."updated_at", "posts"."reply_to_post_number", "posts"."reply_count", "posts"."quote_count", "posts"."deleted_at", "posts"."off_topic_count", "posts"."like_count", "posts"."incoming_link_count", "posts"."bookmark_count", "posts"."score", "posts"."reads", "posts"."post_type", "posts"."sort_order", "posts"."last_editor_id", "posts"."hidden", "posts"."hidden_reason_id", "posts"."notify_moderators_count", "posts"."spam_count", "posts"."illegal_count", "posts"."inappropriate_count", "posts"."last_version_at", "posts"."user_deleted", "posts"."reply_to_user_id", "posts"."percent_rank", "posts"."notify_user_count", "posts"."like_score", "posts"."deleted_by_id", "posts"."edit_reason", "posts"."word_count", "posts"."version", "posts"."cook_method", "posts"."wiki", "posts"."baked_at", "posts"."baked_version", "posts"."hidden_at", "posts"."self_edits", "posts"."reply_quoted", "posts"."via_email", "posts"."raw_email", "posts"."public_version", "posts"."action_code", "posts"."locked_by_id", "posts"."image_upload_id", (TS_RANK_CD(
  post_search_data.search_data,
  TO_TSQUERY('english', '''test'':*ABCD'),
  0|32
)
 * (
  CASE categories.search_priority
  WHEN 2
  THEN 0.6
  WHEN 3
  THEN 0.8
  WHEN 4
  THEN 1.2
  WHEN 5
  THEN 1.4
  ELSE
    CASE WHEN topics.closed
    THEN 0.9
    ELSE 1
    END
  END
)
) rank, topics.bumped_at topic_bumped_at FROM "posts" INNER JOIN "post_search_data" ON "post_search_data"."post_id" = "posts"."id" INNER JOIN "topics" ON "topics"."id" = "posts"."topic_id" AND ("topics"."deleted_at" IS NULL) LEFT JOIN categories ON categories.id = topics.category_id WHERE ("posts"."deleted_at" IS NULL) AND "posts"."post_type" IN (1, 2, 3) AND (topics.visible) AND (topics.archetype = 'private_message' AND post_search_data.private_message) AND (posts.topic_id IN (SELECT topic_id
FROM topic_allowed_users
WHERE user_id = 99999
UNION ALL
SELECT tg.topic_id
FROM topic_allowed_groups tg
JOIN group_users gu ON gu.user_id = 99999 AND gu.group_id = tg.group_id
)) AND (post_search_data.search_data @@ TO_TSQUERY('english', '''test'':*ABCD')) AND (posts.topic_id IN (SELECT topic_id
FROM topic_allowed_users
WHERE user_id = 99999
UNION ALL
SELECT tg.topic_id
FROM topic_allowed_groups tg
JOIN group_users gu ON gu.user_id = 99999 AND gu.group_id = tg.group_id
)) AND ((categories.id IS NULL) OR (NOT categories.read_restricted) OR (categories.id IN (999999))) ORDER BY rank DESC, topic_bumped_at DESC
```
2020-08-24 13:49:43 +08:00
2f043dc89a Fix lint. 2020-08-24 12:38:46 +08:00
4b30799054 FIX: Correct personal_messages:<username> advanced search filter.
Renamed from `private_messages` to `personal_messages` without
deprecation because the `private_messages` advanced search filter never
worked in the first place when it was implemented.
2020-08-24 11:54:30 +08:00
f51ccea028 FIX: Backups should use relative paths for local uploads
This also ensures that restoring a backup works when it was created with the wrong upload paths in the time between ab4c0a4970163506e2c72884ff2ba2d8845eb10c (shortly after v2.6.0.beta1) and this fix.
2020-08-21 15:22:28 +02:00
106a2f58a2 DEV: Drop support for deprecated in:private search filter. 2020-08-21 17:18:39 +08:00
0684118008 DEV: Remove array_agg from search orders that does not need it. 2020-08-21 14:39:07 +08:00
713e6f1bb6 Version bump to v2.6.0.beta2 2020-08-20 16:36:25 -04:00
208d85aaff DEV: Ensure prettier uses the same patterns everywhere 2020-08-20 16:27:32 +02:00
11647b79f7 FIX: SiteSettings::LocalProcessProvider didn't work on multisite
It always used "test" as current site.
2020-08-20 11:15:20 +02:00
89fcb75af2 FIX: default_tags_muted setting should work for anonymous users too. 2020-08-20 10:40:03 +05:30
4395e4d165 FIX: Sending a PM through a flag on a deleted post
Because we allow all the other flag types on a deleted post we should be
able to send a pm to the user letting them know why we deleted their
post.

Bug report:

https://meta.discourse.org/t/-/161156
2020-08-19 17:36:52 -06:00
8348a41124 FEATURE: add regular_categories field in site setting & user option. (#10477)
Like "default watching" and "default tracking" categories option now the "regular" categories support is added. It will be useful for sites that are muted by default. The user option will be displayed only if `mute_all_categories_by_default` site setting is enabled.
2020-08-20 00:35:04 +05:30
a3c0d4a8b5 FIX: Error message when setting enforce 2fa with social logins (#10479) 2020-08-19 13:16:31 -05:00
367de2594d FIX: Unlike own posts on ownership transfer (#10446)
* FIX: Unlike own posts on ownership transfer

If a user has liked a post that has passed the
`post_undo_action_window_mins` system setting window and you transfer ownership
of that post to that user you will be the owner of a post that you have
liked, but cannot unlike resulting in a weird UI behavior. This commit
fixes this issue.

The existing tests didn't check for the timeout window for unliking
posts so I added that in.

I couldn't find a good way to do this logic inside of the guardian class
so rather than duplicating behavior of the `PostActionDestroyer` class
inside of the `PostOwnerChanger` I decided to pass in a "bypass"
variable that could be used to check if the calling class is the
'post_owner_changer' and bypass the guardian instead. I went this route
because the guardian `can_delete_post_action` method has no way of
distinguishing how to allow a user to be able to unlike their own posts
after the timeout window but only on a post owner change.

* use an options hash instead
2020-08-19 09:21:02 -06:00
aa1fc01307 FEATURE - Moderators can create and manage groups (#10432)
Enabling the moderators_manage_categories_and_groups site setting will allow moderator users to create/manage groups.

* show New Group form to moderators

* Allow moderators to update groups and read logs, where appropriate

* Rename site setting from create -> manage

* improved tests

* Migration should rename old log entries

* Log group changes, even if those changes mean you can no longer see the group

* Slight reshuffle

* RouteTo /g if they no longer have permissions to view group
2020-08-19 10:41:40 -04:00
882b0aac19 DEV: Let themes extend color definitions (#10429)
Themes can now declare custom colors that get compiled in core's color definitions stylesheet, thus allowing themes to better support dark/light color schemes. 

For example, if you need your theme to use tertiary for an element in a light color scheme and quaternary in a dark scheme, you can add the following SCSS to your theme's `color_definitions.scss` file: 

```
:root {
  --mytheme-tertiary-or-quaternary: #{dark-light-choose($tertiary, $quaternary)};
}
```

And then use the `--mytheme-tertiary-or-quaternary` variable as the color property of that element. You can also use this file to add color variables that use SCSS color transformation functions (lighten, darken, saturate, etc.) without compromising your theme's compatibility with different color schemes.
2020-08-18 13:02:13 -04:00
2a7490149c DEV: don't fail if in uploads:fix_missing_s3 when fix fails
Previously a single error on a file like invalid extension could fail the
entire rake task
2020-08-18 17:55:49 +10:00
92b7fe4c62 PERF: Add partial index for non-pm search. 2020-08-18 15:55:08 +08:00
33c21c5921 DEV: Skip readonly tables when remapping content (#10461) 2020-08-18 08:53:12 +01:00
24fe08230f FEATURE: ensure posts are rebaked when missing is fixed
This ensures any corrupt optimized images are removed and re-created
2020-08-18 15:37:24 +10:00
645b65adfa FIX: Add playsinline to twitter GIFs 2020-08-18 00:26:51 +01:00
eb981b8fbb FEATURE: Autoplay oneboxed twitter GIF media 2020-08-17 23:56:41 +01:00
a895e4a15b FIX: Skip rendering twitter video if matching format not found
Followup to d5a4318a
2020-08-17 20:53:14 +01:00
d5a4318ac1 FIX: Replace iframe with <video> for twitter videos
This avoids the samesite cookie related error on chrome. It also adds support for twitter 'GIF' content, and allows videos to resize smoothly for narrow devices.
2020-08-17 20:37:36 +01:00
65649eaef0 User card settings (#10302)
* settings implemented

* prettier

* settings updated

* rubocop

* prettier

* Revert "rubocop"

This reverts commit 7805145a7dc1dbccd9a4378bec3626a81ebaa659.

* Revert "prettier"

This reverts commit 2c53f4fa127a00ee9b3c3ab5b1182036089c286b.

* settings updated and changed

* rubocop

* changes applied

* final changes done

* Server side feature added

* spec changed

* changed user_updater and profile file

* Fix user card specs

* web hook serializer solved

* site-setting changed

Co-authored-by: Mark VanLandingham <markvanlan@gmail.com>
2020-08-17 12:37:45 -04:00
0a5376084d FIX: Ensure auto close notice is posted with system locale
Previously it was created with the locale of the user who created the final post in the topic
2020-08-17 15:40:47 +01:00
248bebb8cd PERF: Remove extra subquery in search.
I also noticed that removing the subquery helps the planner to plan
better.
2020-08-17 13:52:12 +08:00
8ac85f54fb DEV: Remove blob: workers from CSP (#10440)
Ace editor is reconfigured to load workers directly from their JS URL. Workers must be on the same origin as the site, so they will not use the CDN.
2020-08-14 18:15:30 +01:00
476d26159a FEATURE: add new user option skip_new_user_tips. (#10437)
And add new site setting `default_other_skip_new_user_tips` in user preferences category.
2020-08-14 19:10:56 +05:30
3cf93e9a8f Revert "DEV: Disable ACE editor worker blobs"
This reverts commit d5463d2a4d7999c5157e1f2db0b10f67120e660c.

With S3 assets, CORS prevents loading worker assets directly.
2020-08-14 14:12:11 +01:00
a5608025aa FIX: Add script asset locations to worker-src CSP directives
We no longer need :blob worker src since d5463d2a. But we do want to allow workers to be loaded from all our existing script-src options.
2020-08-14 12:31:50 +01:00
4670b62969 DEV: IMAP log to database (#10435)
Convert all IMAP logging to write to a database table for easier inspection. These logs are cleaned up daily if they are > 5 days old.

Logs can easily be watched in dev by setting DISCOURSE_DEV_LOG_LEVEL=\"debug\" and running tail -f development.log | grep IMAP
2020-08-14 12:01:31 +10:00
eae8b0465c DEV: Upgrade ACE Editor to 1.4.12, remove extra ACE files (#10431) 2020-08-13 21:53:27 -04:00
ffb31b8d2b FIX: Do not require tagging to be enabled for IMAP archive and delete (#10426)
Previously we did an early return if either SiteSetting.tagging_enabled or SiteSetting.allow_staff_to_tag_pms was false when updating the email on the IMAP server -- however this also stopped us from archiving or deleting emails if either of these were disabled.
2020-08-13 14:04:40 +10:00
5be26c7d24 DEV: add error handling in case download fails 2020-08-13 13:48:23 +10:00