Commit Graph

4871 Commits

Author SHA1 Message Date
f9d55b4941 FEATURE: Update the topic excerpt when the OP is rebaked (#9852)
* We now have a site setting "topic_excerpt_maxlength" that is used when the OP is created or revised to generate a topic excerpt.
* However, posts created before this setting was introduced cannot benefit from this change unless they are revised, and if the topic excerpt length setting is changed that situation is also not covererd.
* This PR makes a change to rebake! to update the topic excerpt IF the post is the OP.
2020-05-22 13:04:15 +10:00
c0779df99d FIX: Remove access control post FK from uploads (#9853) 2020-05-22 11:20:25 +10:00
df68d11c38 FEATURE: Add topic excerpt max length site setting (#9847)
Adds a new topic_excerpt_maxlength site setting.

* When topic excerpt is requested for a post, use the new topic_excerpt_maxlength site setting to limit the size of the excerpt
* Remove code for getting/setting Post.excerpt_size as it is not used anywhere
2020-05-21 13:19:48 +10:00
096eca0ee8 FIX: Requests were not being logged correctly
`enable` was defaulting to `nil` which is not what we wanted.
2020-05-20 15:09:17 -04:00
8851b79472 DEV: less code for Category#url 2020-05-20 18:01:06 +02:00
fb15da43da Remove old web hooks in favor of 'reviewable' web hook (#9776)
* FIX: Emit web hooks for flags

* FEATURE: Remove 'flag' web hook in favor of 'reviewable' web hook

* FEATURE: Remove 'queued post' web hook in favor of 'reviewable' web hook

* FIX: Do not set a default value for web hooks with no events
2020-05-20 12:07:48 +03:00
f7f436e536 DEV: Install rails_failover gem to test our Redis changes. 2020-05-20 15:40:27 +08:00
234cd5c3e7 FIX: Switch discobot to pull avatar from gravatar. 2020-05-20 10:20:08 +08:00
6b92c78033 FIX: purge all associated data on user delete
This commit reorganises the delete dependencies on users and make sure
all are covered. We forgot some on bookmarks, security keys, anon users
and so on.
2020-05-20 11:27:57 +10:00
05ca0c908f FIX: GroupArchivedMessage belongs to Group, not User 2020-05-19 14:13:28 +03:00
725e38f9d7 DEV: Allow plugins to request topic thumbnail sizes (#9828)
In plugin.rb, you can register new sizes like

```
register_topic_thumbnail_size [512, 512]
```

For more information about thumbnails see 03818e642a
2020-05-19 10:38:58 +01:00
6d5e9db883 FIX: Restoring backup didn't clear cached translation overrides 2020-05-18 18:51:51 +02:00
27c611b1d1 FIX: Reverting multiple translation overrides didn't clear cache 2020-05-18 18:51:29 +02:00
f968155288 FIX: Ensure no image downloads during topic thumbnail serialization
If image uploads are missing a width/height, we re-download them to update the width/height in the database. However, this takes significant resources so we don't want to run this logic during topic serialization.
2020-05-18 12:04:29 +01:00
d01c336899 DEV: Clean up some Redis leaks in test env. 2020-05-18 17:27:37 +08:00
461b4e5cc6 DEV: Add framework for filtered plugin registers (#9763)
* DEV: Add framework for filtered plugin registers

Plugins often need to add values to a list, and we need to filter those lists at runtime to ignore values from disabled plugins. This commit provides a re-usable way to do that, which should make it easier to add new registers in future, and also reduce repeated code.

Follow-up commits will migrate existing registers to use this new system

* DEV: Migrate user and group custom field APIs to plugin registry

This gives us a consistent system for checking plugin enabled state, so we are repeating less logic. API changes are backwards compatible
2020-05-15 14:04:38 +01:00
0495a748d0 FIX: Use CDN URLs for topic thumbnails 2020-05-15 13:35:20 +01:00
0cbaa8d813 FEATURE: extend duration allowed for download
Previously we would raise a warning in the logs if downloading
a file (from s3) takes longer than 60 seconds.

At scale this happens reasonably frequently.

1. Raised the duration to 3 minutes

2. Pulled the resizing mutex out of the downloading mutex
so we have less and clearer error logs
2020-05-15 12:45:47 +10:00
165fa13c9d FIX: Always return 0 for non-human user current DraftSequence. 2020-05-15 10:40:36 +08:00
869f9b20a2 PERF: Dematerialize topic_reply_count (#9769)
* PERF: Dematerialize topic_reply_count

It's only ever used for trust level promotions that run daily, or compared to 0. We don't need to track it on every post creation.

* UX: Add symbol in TL3 report if topic reply count is capped

* DEV: Drop user_stats.topic_reply_count column
2020-05-14 15:42:00 -07:00
bec6bace53 DEV: Allow plugins to hide user stats by new guardian method (#9772) 2020-05-14 11:57:35 -05:00
1059557ce1 DEV: Standardize ignored_columns removal comments (#9771) 2020-05-13 13:08:15 -07:00
7f841dc21f PERF: avoid race conditions when creating topic links
Previously the code was very race condition prone leading to
odd failures in production

It was re-written in raw SQL to avoid conditions where rows
conflict on inserts

There is no clean way in ActiveRecord to do:

Insert, on conflict do nothing and return existing id.

This also increases test coverage, we were previously not testing
the code responsible for crawling external sites directly
2020-05-13 16:05:57 +10:00
34c39b65bb PERF: Avoid executing the same query twice. 2020-05-13 11:10:09 +08:00
9981fa4466 FIX: Prevent column name conflicts in reviewable code (#9753)
We were getting errors like this in Reviewables in some cases:

```
ActiveRecord::StatementInvalid (PG::AmbiguousColumn: ERROR:  column reference "category_id" is ambiguous
LINE 4: ...TRUE) OR (reviewable_by_group_id IN (NULL))) AND (category_i...
```

The problem that was making everything go boom is that plugins can add their own custom filters for Reviewables. If one is doing an INNER JOIN on topics, which has its own category_id column, we would get the above AmbiguousColumn error. The solution here is to just make all references to the reviewable columns in the list_for and viewable_by code prefixed by the table name e.g. reviewables.category_id.
2020-05-13 09:05:56 +10:00
497dc6eaa7 Add a global setting for CDN origin
This is so that, on a multisite cluster, when we handle a CDN request,
the hostname that is requested corresponds to one of the sites -
specifically the default site.
2020-05-12 16:43:40 +01:00
088c11a12c FIX: Don't raise an exception if a TopicLink cannot be created
This can happen under concurrency and it's fine. If a `TopicLink`
already exists we can continue on without raising an error.
2020-05-12 11:15:59 -04:00
a29ae17d3a FIX: saving drafts unconditionally increases sequence
Previously we only changed sequence on ownership change, this
cause a race condition between tabs where user could type for a
long time without being warned of an out of date draft.

This change is a radical change and we should watch closely.

Code was already in place to track sequence on the client so no
changes are needed there.
2020-05-12 16:55:42 +10:00
8fb99f218d FEATURE: Allow plugins to register a callback to ignore DraftSequence. 2020-05-12 14:25:20 +08:00
d301af39bd PERF: Avoid hitting DB when fetching draft sequence of bot user. 2020-05-12 13:56:58 +08:00
8149bfbaf1 FEATURE: Filter settings by plugin (#9692) 2020-05-10 14:07:45 +03:00
b2e4ca968b DEV: Remove translate API from plugin.
It appears to be completely unused at this point.
2020-05-08 14:14:01 -04:00
efd0e4d02b FIX: Don't trigger user_updated event for bot users. 2020-05-08 09:28:16 +08:00
37be2958f7 DEV: Allow specifying if a notification is high_priority on create (#9660)
This allows for special cases where we may not want a certain notification type to ALWAYS be high priority (e.g. a topic timer).
2020-05-07 14:35:32 +10:00
6fb0f36ce1 FEATURE: Optionally delete bookmark when reminder sent (#9637)
We now show an options gear icon next to the bookmark name.

When expanded we show the "delete bookmark when reminder sent" option. The value of this checkbox is saved in local storage for the user.

If this is ticked, when a reminder is sent for the bookmark the bookmark itself is deleted. This is so people can use the reminder functionality by itself.

Also remove the blue alert reminder section from the "Edit Bookmark" modal as it just added clutter, because the user can already see they had a reminder set:

Adds a default false boolean column `delete_when_reminder_sent` to bookmarks.
2020-05-07 13:37:39 +10:00
f5bdbd347e Revert "FIX: Discobot has not been created with our custom avatar."
This reverts commit 1062dbc3e9cbf9370803c0acea55eb3c054060ca.

Looks like it's causing some errors on migration.

```
/var/www/discourse/lib/image_sizer.rb:6:in `resize'
/var/www/discourse/lib/upload_creator.rb:120:in `block in create_for'
/var/www/discourse/lib/distributed_mutex.rb:33:in `block in synchronize'
/var/www/discourse/lib/distributed_mutex.rb:29:in `synchronize'
/var/www/discourse/lib/distributed_mutex.rb:29:in `synchronize'
/var/www/discourse/lib/distributed_mutex.rb:14:in `synchronize'
/var/www/discourse/lib/upload_creator.rb:37:in `create_for'
```
2020-05-06 11:28:42 -04:00
fa572d3a7a DEV: Remove all code referencing at_desktop bookmark reminders (#9650)
We have found no need for these reminder types, so we are removing the code for them.
2020-05-06 15:22:43 +10:00
1062dbc3e9 FIX: Discobot has not been created with our custom avatar.
Previously the image was imported from a Discourse hosted CDN but the
URL has since become invalid. However, it was not caught since all
errors are rescued. This commit fixes the issue by shipping the user
avatar with the plugin.
2020-05-06 09:25:01 +08:00
612284cef3 DEV: Remove Discourse.RAW_TEMPLATES (#9630)
We were sharing `Discourse` both as an application object and a
namespace which complicated things for Ember CLI. This patch
moves raw templates into `__DISCOURSE_RAW_TEMPLATES` and adds
a couple helper methods to create/remove them.
2020-05-05 12:15:03 -04:00
03818e642a FEATURE: Include optimized thumbnails for topics (#9215)
This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this:

- Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact

- A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake

- Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued

- Topic lists and topics now include a `thumbnails` key, which includes all the available images:
   ```
   "thumbnails": [
   {
     "max_width": null,
     "max_height": null,
     "url": "//example.com/original-image.png",
     "width": 1380,
     "height": 1840
   },
   {
     "max_width": 1024,
     "max_height": 1024,
     "url": "//example.com/optimized-image.png",
     "width": 768,
     "height": 1024
   }
   ]
  ```

- Themes can request additional thumbnail sizes by using a modifier in their `about.json` file:
   ```
    "modifiers": {
      "topic_thumbnail_sizes": [
        [200, 200],
        [800, 800]
      ],
      ...
  ```
  Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated

- Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 09:07:50 +01:00
9bff0882c3 FEATURE: Nokogumbo (#9577)
* FEATURE: Nokogumbo

Use Nokogumbo HTML parser.
2020-05-05 13:46:57 +10:00
a51b8d9c66 FIX: Do not use cached settings during theme compilation
We compile within a database transaction, so using a cached value from redis can cause unwanted side effects
2020-05-04 09:43:06 +01:00
4f885d7da2 FIX: Clear theme caches after database transaction has committed
This was causing unusual side effects on high-traffic sites, where the cache would be rebuilt before the transaction had been committed
2020-05-04 09:42:50 +01:00
784bf2a173 FIX: Unassign user titles when a badge is deleted (#9573) 2020-05-02 18:02:28 -07:00
37e93914fc FIX: the muted message should be sent after edit (#9593)
Recently, we added feature that we are sending `/muted` to users who muted specific topic just before `/latest` so the client knows to ignore those messages - https://github.com/discourse/discourse/pull/9482

Same `/muted` message should be included when the post is edited
2020-05-01 08:33:57 +10:00
2cb9e85d14 FEATURE: add category banner for why a user cannot post (#9576)
* FEATURE: add category banner for why a user cannot post

Adds a category banner for why a user is unable to post in a category.

Also adds an extra alert for the user when a user is unable to create a topic in a
category and they still try and click on the disabled-looking new topic
button.
2020-04-30 10:39:11 -07:00
d0d5a138c3 DEV: stop freezing frozen strings
We have the `# frozen_string_literal: true` comment on all our
files. This means all string literals are frozen. There is no need
to call #freeze on any literals.

For files with `# frozen_string_literal: true`

```
puts %w{a b}[0].frozen?
=> true

puts "hi".frozen?
=> true

puts "a #{1} b".frozen?
=> true

puts ("a " + "b").frozen?
=> false

puts (-("a " + "b")).frozen?
=> true
```

For more details see: https://samsaffron.com/archive/2018/02/16/reducing-string-duplication-in-ruby
2020-04-30 16:48:53 +10:00
310a7edee5 DEV: remove unused columns from posts and topics
avg_time on posts and topics have not been used in a year.

This uses a re-runnable ddl transaction diasabled migration to
drop the column, cause it touchs very high traffic table and may
deadlock
2020-04-30 11:22:33 +10:00
9d781d4959 FIX: error customizing text for badges from plugins 2020-04-28 14:34:41 -04:00
198856f78d DEV: annotate models 2020-04-28 15:59:39 +05:30