Commit Graph

8018 Commits

Author SHA1 Message Date
f9d55b4941 FEATURE: Update the topic excerpt when the OP is rebaked (#9852)
* We now have a site setting "topic_excerpt_maxlength" that is used when the OP is created or revised to generate a topic excerpt.
* However, posts created before this setting was introduced cannot benefit from this change unless they are revised, and if the topic excerpt length setting is changed that situation is also not covererd.
* This PR makes a change to rebake! to update the topic excerpt IF the post is the OP.
2020-05-22 13:04:15 +10:00
d11c462104 SECURITY: ERB execution in custom Email Style 2020-05-21 14:48:30 -04:00
f4b82f1dc0 DEV: Fix randomly failing spec.
If a user is created with an id of 999, a `upload.user_id ==
user_avatar.user_id` will return true. This fix increases the id of the
upload to something that we will not hit in the foreseeable future.
2020-05-21 11:41:07 +08:00
df68d11c38 FEATURE: Add topic excerpt max length site setting (#9847)
Adds a new topic_excerpt_maxlength site setting.

* When topic excerpt is requested for a post, use the new topic_excerpt_maxlength site setting to limit the size of the excerpt
* Remove code for getting/setting Post.excerpt_size as it is not used anywhere
2020-05-21 13:19:48 +10:00
68db5deaec FIX: Badge granter was disabled by default. 2020-05-21 06:55:22 +08:00
096eca0ee8 FIX: Requests were not being logged correctly
`enable` was defaulting to `nil` which is not what we wanted.
2020-05-20 15:09:17 -04:00
e1bd57007b DEV: Specs for AboutController can_see_about_stats? impact (#9843) 2020-05-20 11:30:27 -05:00
02f44def56 FIX: Don't blow up when trying to parse invalid or non-ASCII URLs (#9838)
* FIX: Don't blow up when trying to parseinvalid or non-ASCII URLs

Follow-up to 72f139191e
2020-05-20 12:46:27 +03:00
fb15da43da Remove old web hooks in favor of 'reviewable' web hook (#9776)
* FIX: Emit web hooks for flags

* FEATURE: Remove 'flag' web hook in favor of 'reviewable' web hook

* FEATURE: Remove 'queued post' web hook in favor of 'reviewable' web hook

* FIX: Do not set a default value for web hooks with no events
2020-05-20 12:07:48 +03:00
72f139191e FIX: S3 store has_been_uploaded? was not taking into account s3 bucket path (#9810)
In some cases, between Discourse forums the hostname of a URL could match if they are hosting S3 files on the same bucket but the S3 bucket path might not. So e.g. https://testbucket.somesite.com/testpath/some/file/url.png vs https://testbucket.somesite.com/prodpath/some/file/url.png. So has_been_uploaded? was returning true for the second URL, even though it may have been uploaded on a different Discourse forum.

This is a very rare case but must be accounted for, because this impacts UrlHelper.is_local which mistakenly thinks the file has already been downloaded and thus allows the URL to be cooked, where we want to return the full URL to be downloaded using PullHotlinkedImages.
2020-05-20 10:40:38 +10:00
e9f579bd3f DEV: Clean up thumbnail tests correctly 2020-05-19 11:04:02 +01:00
725e38f9d7 DEV: Allow plugins to request topic thumbnail sizes (#9828)
In plugin.rb, you can register new sizes like

```
register_topic_thumbnail_size [512, 512]
```

For more information about thumbnails see 03818e642a
2020-05-19 10:38:58 +01:00
96c02caba7 DEV: Change use of Redis flushall to flushdb.
FLUSHALL removes all keys from all databases. Instead we only want to
remove keys from the current Redis database.
2020-05-19 10:20:00 +08:00
f99f6ca111 FIX: randomize file name when created from fixtures (#9731)
* FIX: randomize file name when created from fixtures

When a temporary file is created from fixtures it should have a unique name.
It is to prevent a collision in parallel specs evaluation

* FIX: use /tmp/pid folder to keep fixture files
2020-05-19 09:09:36 +10:00
6d5e9db883 FIX: Restoring backup didn't clear cached translation overrides 2020-05-18 18:51:51 +02:00
27c611b1d1 FIX: Reverting multiple translation overrides didn't clear cache 2020-05-18 18:51:29 +02:00
96848b7649 UX: Allow secure media URLs to be cached for a short period of time
Signed S3 URLs are valid for 15 seconds, so we can safely allow the browser to cache them for 10 seconds. This should help with large numbers of requests when composing a post with many images.
2020-05-18 15:00:41 +01:00
918bd75909 FIX: allows to remove a topic timer with delete_replies as type (#9800) 2020-05-18 11:47:08 +02:00
d01c336899 DEV: Clean up some Redis leaks in test env. 2020-05-18 17:27:37 +08:00
862773ec83 FIX: do not remove stop words when using English locale
PG already handles English stop words, the list in cppjieba is
bigger than the list PG uses, which in turn causes confusion cause
words such as "volume" are stripped using cppijieba stop word list

We will follow up with another commit here to apply the Chinese
word stopwords, but for now to eliminate the confusion we are
skipping applying the stopword list when the dictionary in PG is
in English.
2020-05-18 10:54:56 +10:00
ea63fa7de7 FIX: extract reset_last_seen_cache! in user_spec (#9808)
Helper method to avoid code duplication
2020-05-18 09:13:37 +10:00
461b4e5cc6 DEV: Add framework for filtered plugin registers (#9763)
* DEV: Add framework for filtered plugin registers

Plugins often need to add values to a list, and we need to filter those lists at runtime to ignore values from disabled plugins. This commit provides a re-usable way to do that, which should make it easier to add new registers in future, and also reduce repeated code.

Follow-up commits will migrate existing registers to use this new system

* DEV: Migrate user and group custom field APIs to plugin registry

This gives us a consistent system for checking plugin enabled state, so we are repeating less logic. API changes are backwards compatible
2020-05-15 14:04:38 +01:00
0495a748d0 FIX: Use CDN URLs for topic thumbnails 2020-05-15 13:35:20 +01:00
858735b5da DEV: Fix broken test due to 9ab5801a1bc5bfc505673d0c74f1a3c1e6c28461. 2020-05-15 15:00:04 +08:00
9ab5801a1b FIX: Allow post migrations using #change to carry out unsafe migration 2020-05-15 14:23:27 +08:00
165fa13c9d FIX: Always return 0 for non-human user current DraftSequence. 2020-05-15 10:40:36 +08:00
1a2b9435b0 DEV: Standardize table sorting verbiage (#9757)
* DEV: Standardize table sorting verbiage

This commit creates a common component that tables can use to make their
headers sortable. This commit also standardizes on using `desc` as the
default and passing in the `asc=true` flag to adjust the sorting
direction.

* Add deprecation warnings

Adds deprecation warnings if using previous params and maintains
backwards compatibility. Set the default sort value for group members to
be asc.

* switch group requests to use common table-header-toggle

* update fixture
2020-05-14 20:10:59 -06:00
65a3fdbc57 FIX: randomly failing user_spec (#9754)
* FIX: randomly falling user_spec

When we evaluate `update_last_seen!` we relay on Redis to not run that code too often

https://github.com/discourse/discourse/blob/master/app/models/user.rb#L753

The problem is that not all specs which are running `update_last_seen!` are not cleaning after themselves

For examples specs in that block https://github.com/discourse/discourse/blob/master/spec/models/user_spec.rb#L901

So it can be replicated when you run a few times
`bundle exec rspec  ./spec/models/user_spec.rb -e "should not update the first seen value if it doesn't exist" -e "should have 0 for days_visited"`

We should delete Redis key after each spec which is evaluating `update_last_seen!`
2020-05-15 09:24:07 +10:00
869f9b20a2 PERF: Dematerialize topic_reply_count (#9769)
* PERF: Dematerialize topic_reply_count

It's only ever used for trust level promotions that run daily, or compared to 0. We don't need to track it on every post creation.

* UX: Add symbol in TL3 report if topic reply count is capped

* DEV: Drop user_stats.topic_reply_count column
2020-05-14 15:42:00 -07:00
5ff2a235f6 DEV: Allow 3-digit HEX color code in single icon route
Followup to aee8e62
2020-05-14 16:37:45 -04:00
aee8e62e21 FEATURE: Add endpoint for individual SVG icons (#9765) 2020-05-14 14:17:19 -04:00
a047004c9a FIX: Specific email error for replies to digest emails (#9770) 2020-05-14 09:04:58 -05:00
3ed6a0e904 FIX: Detect Wayback Machine using user agent (#9777) 2020-05-14 21:10:07 +10:00
d4717f5d1e DEV: Reduce repeated code in discourse_plugin_registry (#9705)
Use a helper method to simplify creating a new register. Previously this would require creating lots of different methods manually, and adding every register to the clear/reset functions
2020-05-13 12:25:34 +01:00
7f841dc21f PERF: avoid race conditions when creating topic links
Previously the code was very race condition prone leading to
odd failures in production

It was re-written in raw SQL to avoid conditions where rows
conflict on inserts

There is no clean way in ActiveRecord to do:

Insert, on conflict do nothing and return existing id.

This also increases test coverage, we were previously not testing
the code responsible for crawling external sites directly
2020-05-13 16:05:57 +10:00
a64cf265fd FIX: Change /bookmarks URL back to topic list and add bookmark poster avatars (#9759)
* Changes the /bookmarks URL to show the original "topics filtered by bookmark" list instead of redirecting to user activity bookmarks (see https://meta.discourse.org/t/domain-com-bookmarks-is-showing-domain-com-u-user-activity-bookmarks-with-reminders/149252/12)
* Add the user avatar for the user who made the post that is bookmarked
2020-05-13 14:03:24 +10:00
4078b22887 FIX: Handle missing provider return sso url
This commit prevents a 500 error from occurring if someone is trying to
setup their discourse instance as a sso provider and they don't pass in
a `return_sso_url` in their payload.
2020-05-12 18:16:50 -06:00
9981fa4466 FIX: Prevent column name conflicts in reviewable code (#9753)
We were getting errors like this in Reviewables in some cases:

```
ActiveRecord::StatementInvalid (PG::AmbiguousColumn: ERROR:  column reference "category_id" is ambiguous
LINE 4: ...TRUE) OR (reviewable_by_group_id IN (NULL))) AND (category_i...
```

The problem that was making everything go boom is that plugins can add their own custom filters for Reviewables. If one is doing an INNER JOIN on topics, which has its own category_id column, we would get the above AmbiguousColumn error. The solution here is to just make all references to the reviewable columns in the list_for and viewable_by code prefixed by the table name e.g. reviewables.category_id.
2020-05-13 09:05:56 +10:00
033644ea82 DEV: Adds an integrity spec for JS constants 2020-05-12 12:23:36 -04:00
2298e14d38 FEATURE: Support for App Shortcuts Menu (#9749)
* FEATURE: Support for App Shortcuts Menu

This adds a list of shortcuts to a installed Discourse instance.

It can be accessed by right clicks or long press on the app icon.

See https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/master/Shortcuts/explainer.md

List of possible follow ups include:

- Making it admin customizable
- Making it user customizable
- Using SVG icons from the site icon sprite
- Picking an accent color for icons

* FIX: Add type to shortcut menu icons
2020-05-12 12:24:33 -03:00
c94e6a9a66 FEATURE: add noindex header to tags pages (#9748) 2020-05-12 10:44:46 -04:00
6230f5c554 FEATURE: Allow parameter authentication for UserApiKeys (#9742)
This refactors default_current_user_provider in a few ways:
- Introduce a generic `api_parameter_allowed?` method which checks for whitelisted routes/formats
- Only read the api_key parameter on allowed routes. It is now completely ignored on other routes (previously it would raise a 403)
- Start reading user_api_key parameter on allowed routes
- Refactor tests as end-end integration tests

A plugin API for PARAMETER_API_PATTERNS will be added soon
2020-05-12 13:35:36 +01:00
a29ae17d3a FIX: saving drafts unconditionally increases sequence
Previously we only changed sequence on ownership change, this
cause a race condition between tabs where user could type for a
long time without being warned of an out of date draft.

This change is a radical change and we should watch closely.

Code was already in place to track sequence on the client so no
changes are needed there.
2020-05-12 16:55:42 +10:00
d301af39bd PERF: Avoid hitting DB when fetching draft sequence of bot user. 2020-05-12 13:56:58 +08:00
3c423faafd DEV: Avoid repeating draft key logic in multiple places. 2020-05-12 11:14:03 +08:00
4c8bece104 FEATURE: default canonical URL (#9738)
For pages that do not specify canonical URL we will default to `https://SITENAME/PATH`. 

This ensures that if a URL is crawled on the CDN the search ranking will transfer to the main site.

Additionally we whitelist the `?page` param
2020-05-12 09:13:20 +10:00
ab919332dc DEV: api documentation updates (#9612)
* DEV: api documentation updates

- Created a script to convert json responses to rswag
- Documented several api endpoints
- Switched rswag to use header based auth

* Update script, fix some schema missmatches
2020-05-11 13:06:49 -06:00
5fc51ed49c DEV: Remove unused DiscoursePlugin class (#9715) 2020-05-11 15:46:54 +01:00
07b728c5e5 FEATURE: add noindex header to badges, groups, and /my pages (#9736) 2020-05-11 15:05:42 +10:00
bb4e8899c4 FEATURE: let Google index pages so it can remove them
Google insists on indexing pages so it can figure out if they
can be removed from the index.

see: https://support.google.com/webmasters/answer/6332384?hl=en

This change ensures the we have special behavior for Googlebot
where we allow indexing, but block the actual indexing via
X-Robots-Tag
2020-05-11 12:15:18 +10:00