Commit Graph

38510 Commits

Author SHA1 Message Date
93ff54e184 FIX: improvements for vanilla bulk import (#10212)
Adjustments to the base:
1. PG connection doesn't require host - it was broken on import droplet
2. Drop `topic_reply_count` - it was removed here - https://github.com/discourse/discourse/blob/master/db/post_migrate/20200513185052_drop_topic_reply_count.rb
3. Error with `backtrace.join("\n")` -> `e.backtrace.join("\n")`
4. Correctly link the user and avatar to quote block

Adjustments to vanilla:
1. Top-level Vanilla categories are valid categories
2. Posts have `format` column which should be used to decide if the format is HTML or Markdown
3. Remove no UTF8 characters
4. Remove not supported HTML elements like `font` `span` `sub` `u`
2020-07-14 15:58:27 +10:00
cc01297f1f DEV: Bump rubocop from 0.87.1 to 0.88.0 (#10226)
Bumps [rubocop](https://github.com/rubocop-hq/rubocop) from 0.87.1 to 0.88.0.
- [Release notes](https://github.com/rubocop-hq/rubocop/releases)
- [Changelog](https://github.com/rubocop-hq/rubocop/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rubocop-hq/rubocop/compare/v0.87.1...v0.88.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-07-14 15:57:46 +10:00
d8c796bc44 FIX: Ensure that aggregating search shows the post with the higest rank.
Previously, we would only take either the `MIN` or `MAX` for
`post_number` during aggregation meaning that the ranking is not
considered.

```
require 'benchmark/ips'

Benchmark.ips do |x|
  x.config(time: 10, warmup: 2)

  x.report("current aggregate search query") do
    DB.exec <<~SQL
    SELECT "posts"."id", "posts"."user_id", "posts"."topic_id", "posts"."post_number", "posts"."raw", "posts"."cooked", "posts"."created_at", "posts"."updated_at", "posts"."reply_to_post_number", "posts"."reply_count", "posts"."quote_count", "posts"."deleted_at", "posts"."off_topic_count", "posts"."like_count", "posts"."incoming_link_count", "posts"."bookmark_count", "posts"."score", "posts"."reads", "posts"."post_type", "posts"."sort_order", "posts"."last_editor_id", "posts"."hidden", "posts"."hidden_reason_id", "posts"."notify_moderators_count", "posts"."spam_count", "posts"."illegal_count", "posts"."inappropriate_count", "posts"."last_version_at", "posts"."user_deleted", "posts"."reply_to_user_id", "posts"."percent_rank", "posts"."notify_user_count", "posts"."like_score", "posts"."deleted_by_id", "posts"."edit_reason", "posts"."word_count", "posts"."version", "posts"."cook_method", "posts"."wiki", "posts"."baked_at", "posts"."baked_version", "posts"."hidden_at", "posts"."self_edits", "posts"."reply_quoted", "posts"."via_email", "posts"."raw_email", "posts"."public_version", "posts"."action_code", "posts"."locked_by_id", "posts"."image_upload_id" FROM "posts" JOIN (SELECT *, row_number() over() row_number FROM (SELECT topics.id, min(posts.post_number) post_number FROM "posts" INNER JOIN "post_search_data" ON "post_search_data"."post_id" = "posts"."id" INNER JOIN "topics" ON "topics"."id" = "posts"."topic_id" AND ("topics"."deleted_at" IS NULL) LEFT JOIN categories ON categories.id = topics.category_id WHERE ("posts"."deleted_at" IS NULL) AND "posts"."post_type" IN (1, 2, 3, 4) AND (topics.visible) AND (topics.archetype <> 'private_message') AND (post_search_data.search_data @@ TO_TSQUERY('english', '''postgres'':*ABCD')) AND (categories.id NOT IN (
      SELECT categories.id WHERE categories.search_priority = 1
    )
    ) AND ((categories.id IS NULL) OR (NOT categories.read_restricted)) GROUP BY topics.id ORDER BY MAX((
      TS_RANK_CD(
        post_search_data.search_data,
        TO_TSQUERY('english', '''postgres'':*ABCD'),
        1|32
      ) *
      (
        CASE categories.search_priority
        WHEN 2
        THEN 0.6
        WHEN 3
        THEN 0.8
        WHEN 4
        THEN 1.2
        WHEN 5
        THEN 1.4
        ELSE
          CASE WHEN topics.closed
          THEN 0.9
          ELSE 1
          END
        END
      )
    )
    ) DESC, topics.bumped_at DESC LIMIT 51 OFFSET 0) xxx) x ON x.id = posts.topic_id AND x.post_number = posts.post_number WHERE ("posts"."deleted_at" IS NULL) ORDER BY row_number;
    SQL
  end

  x.report("current aggregate search query with proper ranking") do
    DB.exec <<~SQL
    SELECT "posts"."id", "posts"."user_id", "posts"."topic_id", "posts"."post_number", "posts"."raw", "posts"."cooked", "posts"."created_at", "posts"."updated_at", "posts"."reply_to_post_number", "posts"."reply_count", "posts"."quote_count", "posts"."deleted_at", "posts"."off_topic_count", "posts"."like_count", "posts"."incoming_link_count", "posts"."bookmark_count", "posts"."score", "posts"."reads", "posts"."post_type", "posts"."sort_order", "posts"."last_editor_id", "posts"."hidden", "posts"."hidden_reason_id", "posts"."notify_moderators_count", "posts"."spam_count", "posts"."illegal_count", "posts"."inappropriate_count", "posts"."last_version_at", "posts"."user_deleted", "posts"."reply_to_user_id", "posts"."percent_rank", "posts"."notify_user_count", "posts"."like_score", "posts"."deleted_by_id", "posts"."edit_reason", "posts"."word_count", "posts"."version", "posts"."cook_method", "posts"."wiki", "posts"."baked_at", "posts"."baked_version", "posts"."hidden_at", "posts"."self_edits", "posts"."reply_quoted", "posts"."via_email", "posts"."raw_email", "posts"."public_version", "posts"."action_code", "posts"."locked_by_id", "posts"."image_upload_id" FROM "posts" JOIN (SELECT *, row_number() over() row_number FROM (SELECT subquery.topic_id id, (ARRAY_AGG(subquery.post_number))[1] post_number, MAX(subquery.rank) rank, MAX(subquery.bumped_at) bumped_at FROM (SELECT "posts"."id", "posts"."user_id", "posts"."topic_id", "posts"."post_number", "posts"."raw", "posts"."cooked", "posts"."created_at", "posts"."updated_at", "posts"."reply_to_post_number", "posts"."reply_count", "posts"."quote_count", "posts"."deleted_at", "posts"."off_topic_count", "posts"."like_count", "posts"."incoming_link_count", "posts"."bookmark_count", "posts"."score", "posts"."reads", "posts"."post_type", "posts"."sort_order", "posts"."last_editor_id", "posts"."hidden", "posts"."hidden_reason_id", "posts"."notify_moderators_count", "posts"."spam_count", "posts"."illegal_count", "posts"."inappropriate_count", "posts"."last_version_at", "posts"."user_deleted", "posts"."reply_to_user_id", "posts"."percent_rank", "posts"."notify_user_count", "posts"."like_score", "posts"."deleted_by_id", "posts"."edit_reason", "posts"."word_count", "posts"."version", "posts"."cook_method", "posts"."wiki", "posts"."baked_at", "posts"."baked_version", "posts"."hidden_at", "posts"."self_edits", "posts"."reply_quoted", "posts"."via_email", "posts"."raw_email", "posts"."public_version", "posts"."action_code", "posts"."locked_by_id", "posts"."image_upload_id", (
      TS_RANK_CD(
        post_search_data.search_data,
        TO_TSQUERY('english', '''postgres'':*ABCD'),
        1|32
      ) *
      (
        CASE categories.search_priority
        WHEN 2
        THEN 0.6
        WHEN 3
        THEN 0.8
        WHEN 4
        THEN 1.2
        WHEN 5
        THEN 1.4
        ELSE
          CASE WHEN topics.closed
          THEN 0.9
          ELSE 1
          END
        END
      )
    )
     rank, topics.bumped_at bumped_at FROM "posts" INNER JOIN "post_search_data" ON "post_search_data"."post_id" = "posts"."id" INNER JOIN "topics" ON "topics"."id" = "posts"."topic_id" AND ("topics"."deleted_at" IS NULL) LEFT JOIN categories ON categories.id = topics.category_id WHERE ("posts"."deleted_at" IS NULL) AND "posts"."post_type" IN (1, 2, 3, 4) AND (topics.visible) AND (topics.archetype <> 'private_message') AND (post_search_data.search_data @@ TO_TSQUERY('english', '''postgres'':*ABCD')) AND (categories.id NOT IN (
      SELECT categories.id WHERE categories.search_priority = 1
    )
    ) AND ((categories.id IS NULL) OR (NOT categories.read_restricted))) subquery GROUP BY subquery.topic_id ORDER BY rank DESC, bumped_at DESC LIMIT 51 OFFSET 0) xxx) x ON x.id = posts.topic_id AND x.post_number = posts.post_number WHERE ("posts"."deleted_at" IS NULL) ORDER BY row_number;
    SQL
  end

  x.compare!
end
```

```
Warming up --------------------------------------
current aggregate search query
                         1.000  i/100ms
current aggregate search query with proper ranking
                         1.000  i/100ms
Calculating -------------------------------------
current aggregate search query
                         17.726  (± 0.0%) i/s -    178.000  in  10.045107s
current aggregate search query with proper ranking
                         17.802  (± 0.0%) i/s -    178.000  in  10.002230s

Comparison:
current aggregate search query with proper ranking:       17.8 i/s
current aggregate search query:       17.7 i/s - 1.00x  (± 0.00) slower
```
2020-07-14 13:39:13 +08:00
4009c9f711 DEV: Fix search specs to take note of order in assertions.
`contain_exactly` does not care about the order which isn't what we
want.
2020-07-14 13:37:44 +08:00
bcc80e0ea8 FEATURE: Add search to user bookmark list (#10230)
User bookmarks can now be searched by name or post raw content. The q querystring param is hooked up from the Ember router as well.
2020-07-14 14:43:41 +10:00
f4f3e8c401 FIX: Various improvements to bookmark modal UI (#10225)
* Do not autofocus name input on mobile
* Improve code for formatted reminder type times to not be computed, so the modal times update correctly
* Change wording of "Next Monday" to "Monday" for all days except when today is Monday
2020-07-14 14:02:13 +10:00
ce39733b1a FIX: Incorrect search blurb when advanced search filters are used take2
Also remove include_blurbs attribute which isn't used.
2020-07-14 11:50:40 +08:00
919c87a9ce FIX: Get only the correct collapse title in emails (#10228)
Previously, this would get every details block title from the entire post, for every details block.
2020-07-13 18:49:36 -07:00
8ddd45d524 PERF: topic_view participant post count: don't send back ID list (#10210)
On large topics, the cost of sending the entire post ID list back over to the database is signficant. Just have the DB recalculate the list of visible posts instead.
2020-07-13 18:42:09 -07:00
ab77707c1e DEV: Bump oj from 3.10.6 to 3.10.7 (#10227)
Bumps [oj](https://github.com/ohler55/oj) from 3.10.6 to 3.10.7.
- [Release notes](https://github.com/ohler55/oj/releases)
- [Changelog](https://github.com/ohler55/oj/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/ohler55/oj/compare/v3.10.6...v3.10.7)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-07-14 10:57:23 +10:00
277d09516e topic-admin-menu shouldn't be fixed width on mobile 2020-07-13 18:50:57 -04:00
7045a2a87c FIX: Don't strip noopener from oneboxes 2020-07-13 16:54:42 -04:00
c69983a30d Build(deps): Bump multi_json from 1.14.1 to 1.15.0 (#10215)
Bumps [multi_json](https://github.com/intridea/multi_json) from 1.14.1 to 1.15.0.
- [Release notes](https://github.com/intridea/multi_json/releases)
- [Changelog](https://github.com/intridea/multi_json/blob/master/CHANGELOG.md)
- [Commits](https://github.com/intridea/multi_json/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-07-13 15:33:49 -04:00
a73da42691 FIX: Don't award new user of the month in the wrong month
see: https://meta.discourse.org/t/new-user-of-the-month-badge-awarded-before-registraton-date/157347/2?u=eviltrout
2020-07-13 15:05:30 -04:00
c02e358146 FIX: Remove social sharing icons from private contexts (#10213) 2020-07-13 14:35:39 -04:00
e0f97c707e PERF: Cache user summary data 2020-07-13 15:26:56 -03:00
a4d5d1f51d PERF: Add user_id condition so we can use another index in the query 2020-07-13 15:26:56 -03:00
73a3e2ccbc DEV: Add spec for summary links 2020-07-13 15:26:56 -03:00
a900c99993 DEV: Add external_id to notification payload
If sso is enabled the notification payload will now include the
external_id for the user.

This was requested on meta: https://meta.discourse.org/t/-/129052/10
2020-07-13 12:07:39 -06:00
33554e5cbc Build(deps): Bump cose from 1.1.0 to 1.2.0 (#10223)
Bumps [cose](https://github.com/cedarcode/cose-ruby) from 1.1.0 to 1.2.0.
- [Release notes](https://github.com/cedarcode/cose-ruby/releases)
- [Changelog](https://github.com/cedarcode/cose-ruby/blob/master/CHANGELOG.md)
- [Commits](https://github.com/cedarcode/cose-ruby/compare/v1.1.0...v1.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-07-13 12:30:39 -04:00
cf02c518b9 DEV: Merge category and tag hashtags code paths (#10216)
Category and tag hashtags used to be handled differently even though
most of the code was very similar. This design was the root cause of
multiple issues related to hashtags.

This commit reduces the number of requests (just one and debounced
better), removes the use of CSS classes which marked resolved hashtags,
simplifies a lot of the code as there is a single source of truth and
previous race condition fixes are now useless.

It also includes a very minor security fix which let unauthorized users
to guess hidden tags.
2020-07-13 19:13:17 +03:00
c5da813ff5 Build(deps): Bump rubocop-rspec from 1.41.0 to 1.42.0 (#10207)
Bumps [rubocop-rspec](https://github.com/rubocop-hq/rubocop-rspec) from 1.41.0 to 1.42.0.
- [Release notes](https://github.com/rubocop-hq/rubocop-rspec/releases)
- [Changelog](https://github.com/rubocop-hq/rubocop-rspec/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rubocop-hq/rubocop-rspec/compare/v1.41.0...v1.42.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-07-13 12:01:43 -04:00
6c6c56613d FIX: Check if last poster exists before using it
There was a problem when admins viewed topics where the last poster was
deleted.
2020-07-13 18:55:14 +03:00
c8856c7a43 Build(deps): Bump rake-compiler from 1.1.0 to 1.1.1 (#10206)
Bumps [rake-compiler](https://github.com/luislavena/rake-compiler) from 1.1.0 to 1.1.1.
- [Release notes](https://github.com/luislavena/rake-compiler/releases)
- [Changelog](https://github.com/rake-compiler/rake-compiler/blob/master/History.txt)
- [Commits](https://github.com/luislavena/rake-compiler/compare/v1.1.0...v1.1.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-07-13 11:52:56 -04:00
a96ff82e53 FIX: Set the lang/xml:lang html attrs in emails (#10218)
In some rare cases emails are incorrectly sent out with lang placeholders in body (`xml:lang="%{html_lang}"`)
2020-07-13 16:39:40 +02:00
8e2f57a498 Fix the build. 2020-07-13 18:52:04 +05:30
7532f24668 UX: better error message if moderator is not allowed to invite to group
UX: do not show invite to group option if mod is not owner of any group
2020-07-13 18:09:36 +05:30
ba57dc57cc DEV: use group_ids instead of group_names in invite params 2020-07-13 17:37:04 +05:30
eb73048b0f DEV: Allow displaying both title and panels in modals (#10220) 2020-07-13 12:43:06 +02:00
4492718864 FIX: Skip whisper posts when updating topic like count (#10157) 2020-07-13 16:30:00 +10:00
54d002f7db DEV: Fix error introduced in d5c56a846. 2020-07-13 10:13:37 +08:00
753477e684 FIX: Slightly reduce fake delay of discobot user.
Previously it would sleep for either 2 or 3 seconds instead of sleeping
between a range of 2 to 3 seconds. Also, 2 to 3 seconds seems to be
excessivly long when I tried out discobot again.
2020-07-13 09:08:31 +08:00
180494185c FIX: last visit bar regression (#10224)
Last visit regression bar is missing because of that change https://github.com/discourse/discourse/commit/9b7000dbf10

Order property was changed from "default" to null and it was picked by guard condition
2020-07-13 10:42:05 +10:00
6c91f795d0 DEV: widget-dropdown CSS tweaks (#10219)
* DEV: Remove the margin from widget-dropdown

Generic components should not have a margin. Those should be styled in the place where they are used.

* DEV: Remove margin from the dropdown body

It triggered a warning in popper and was effectively a no-op as popper positions dropdowns on its own using `position: fixed` and `top/right/bottom/left` properties.
2020-07-13 01:27:31 +02:00
942cc9b57a DEV: Add rawTitle property support to modal-tab (#10221)
```js
const panels = [
  { id: "test1", rawTitle: "Test 1" },
  { id: "test2", rawTitle: "Test 2" }
];

showModal("a-modal", { panels }));
```
2020-07-13 01:26:45 +02:00
544a9865c6 DEV: Add more API Doc specs for topic endpoints
Added some more specs that will be used to auto generate the api docs.

Added timed topic and update timestamp endpoints.
2020-07-10 12:08:09 -06:00
fbf29443bd FIX: Allow signup when auth provider supplies no email 2020-07-10 17:56:33 +01:00
275b748016 FIX: Replace links to removed uploads from reviewables with a placeholder (#10180) 2020-07-10 17:57:06 +03:00
4b0d86d8d3 FIX: Do not display enabled inputs when username/name are locked 2020-07-10 10:49:32 +01:00
c72bc27888 FEATURE: Implement support for IMAP and SMTP email protocols. (#8301)
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2020-07-10 12:05:55 +03:00
e88b17c044 FIX: Sync client and server side behavior for category hashtag lookup
Category.query_from_hashtag_slug is the only source of truth.
2020-07-10 10:45:39 +03:00
56f42d89c5 FIX: Post menu bookmark icon and attributes not refreshing on notification click (#10214)
When creating a bookmark reminder that deletes the bookmark on reminder, if the user clicked on the notification and got taken to the post in the topic the bookmark icon still showed as blue with the reminder clock indicator. This was because the response JSON for reloading a topic post was not including the bookmark attributes, not even the bookmarked boolean.

We now return the correct attributes in the serializer, and if bookmarked is false we clear all the bookmark related attributes on the post for the notification to make sure nothing of the old bookmark remains in the UI.

This was only a problem if the user did not refresh the app completely inbetween setting the reminder and receiving the notification.
2020-07-10 14:35:53 +10:00
9a2955d471 FIX: Migrate topic_users.bookmarked to false when it is incorrectly true (#10211)
Follow up to https://github.com/discourse/discourse/pull/10188/files

There are still TopicUser records where bookmarked is true even though there are no Bookmark or PostAction records with the type of bookmark for the associated topic and user. This migration corrects this issue by setting bookmarked to false for these cases.
2020-07-10 12:22:15 +10:00
37f7e41e60 DEV: Remove Google+ icon from sprite 2020-07-09 21:29:39 -04:00
706f1a6294 DEV: Remove text-ellipsis component and use CSS line-clamp instead (#10196) 2020-07-09 20:51:43 -04:00
831f3cab56 DEV: Fix search rate limit tests 2020-07-09 20:44:17 -03:00
e74817cbb9 DEV: Document set notification level endpoint
Another commit using rswag to document the api so that the api docs can
be automatically generated.
2020-07-09 17:41:11 -06:00
e866e3d609 FEATURE: Add global rate limit for anon searches (#10208) 2020-07-10 09:08:34 +10:00
79b52b1e9a DEV: Add SVG tests for 31e31ef44 (#10205) 2020-07-09 14:02:25 -07:00
bfde665e76 DEV: Classes and plugin-outlet in admin user-list nav (#10204) 2020-07-09 15:10:25 -05:00