Commit Graph

90 Commits

Author SHA1 Message Date
3c81683955 DEV: Rename UriHelper.escape_uri to .normalized_encode
This is a much better description of its function. It performs idempotent normalization of a URL. If consumers truly need to `encode` a URL (including double-encoding of existing encoded entities), they can use the existing `.encode` method.
2022-08-09 11:55:25 +01:00
f3b2ee8e1b FIX: Use default locale for footer of embedded topics (#17760)
The content from the remote site and the footer get cached for 10 minutes, so Discourse should use the default locale instead of the user locale for the footer. Otherwise Discourse might cache the message in a different language.
2022-08-02 20:49:28 +02:00
357011eb3b DEV: Clean up freedom patches
This patch removes some of our freedom patches that have been deprecated
for some time now.
Some of them have been updated so we’re not shipping code based on an
old version of Rails.
2022-04-06 10:07:14 +02:00
e4e37257cc FIX: Handle malformed URLs in TopicEmbed.absolutize_urls. 2022-01-21 11:18:54 +08:00
cc1b45f58b FIX: Convert URLs embedded topics to absolute form (#14975)
Sometimes the expanded post contained broken relative URLs because they
were not converted to their absolute form.
2021-11-17 16:39:49 +11:00
69f0f48dc0 DEV: Fix rubocop issues (#14715) 2021-10-27 11:39:28 +03:00
2944d2cdd6 FIX: Ruby 3 does not freeze interpolated string (#14567)
Ruby 2.7 or earlier `+contents` returns self.dup
when `frozen_string_literal: true`. However, Ruby 3.0 returns self
because this string is interpolated one, which is not frozen anymore.

This commit uses self.dup to return duplicated string regardless Ruby
versions.
https://bugs.ruby-lang.org/issues/17104
2021-10-11 13:20:18 +11:00
4c2d5158c5 FIX: Follow the canonical URL when importing a remote topic. (#14489)
FinalDestination now supports the `follow_canonical` option, which will perform an initial GET request, parse the canonical link if present, and perform a HEAD request to it.

We use this mode during embeds to avoid treating URLs with different query parameters as different topics.
2021-10-01 12:48:21 -03:00
2e0992c757 DEV: Allow TopicEmbed.import to optionally receive a list of tags (#14301)
This will be used by the rss-polling plugin
2021-09-13 17:01:59 -03:00
ef0471d7ae DEV: Allow passing cook_method to TopicEmbed.import to override default (#14209)
DEV: Allow passing cook_method to TopicEmbed.import to override default

This will be used in the rss-polling plugin when we want to have
oneboxes on feed content, like youtube for example.
2021-09-01 15:46:39 -03:00
560c13211a DEV: Allow passing a category parameter when importing a topic (#14069)
This will be used in the rss pooling plugin to address the feature
request at https://meta.discourse.org/t/-/200644?u=falco
2021-08-17 18:17:07 -03:00
59097b207f DEV: Correct typos and spelling mistakes (#12812)
Over the years we accrued many spelling mistakes in the code base. 

This PR attempts to fix spelling mistakes and typos in all areas of the code that are extremely safe to change 

- comments
- test descriptions
- other low risk areas
2021-05-21 11:43:47 +10:00
ed818a4a19 FIX: prevents malformed href to crash TopicEmbed (#12910)
If the associated page of a remote url passed to `TopicEmber.new(remote_url)` contained a malformed link like: `<a href="(http://foo.bar)">Baz</a>` it would raise an uncaught exception:

```
Job exception: Invalid scheme format: (http
```
2021-04-30 11:10:19 +02:00
27386ba714 FIX: Kernel.open is deprecated (#12387) 2021-03-12 12:42:32 -06:00
3d9eb3e085 Rename options passed to Readability::Document back to whitelist and blacklist (#10340) 2020-07-30 12:56:48 -07:00
e0d9232259 FIX: use allowlist and blocklist terminology (#10209)
This is a PR of the renaming whitelist to allowlist and blacklist to the blocklist.
2020-07-27 10:23:54 +10:00
a41476800b FIX: Don't raise an exception if a topic cannot be retrieved (#9906) 2020-05-28 11:59:20 -03:00
da839e6d26 SECURITY: Use FinalDestination for topic embeds 2020-05-27 09:26:09 -06:00
d9a02d1336 Revert "Revert "Merge branch 'master' of https://github.com/discourse/discourse""
This reverts commit 20780a1eeed56b321daf18ee6bbfe681a51d1bf4.

* SECURITY: re-adds accidentally reverted commit:
  03d26cd6: ensure embed_url contains valid http(s) uri
* when the merge commit e62a85cf was reverted, git chose the 2660c2e2 parent to land on
  instead of the 03d26cd6 parent (which contains security fixes)
2020-05-23 00:56:13 -04:00
20780a1eee Revert "Merge branch 'master' of https://github.com/discourse/discourse"
This reverts commit e62a85cf6fd81a2a34aff6144bd36b9ac459964a, reversing
changes made to 2660c2e21d84bea667e1ea339f91cda352328062.
2020-05-22 20:25:56 -07:00
03d26cd6f0 SECURITY: ensure embed_url contains valid http(s) uri 2020-05-22 14:54:56 -06:00
9bff0882c3 FEATURE: Nokogumbo (#9577)
* FEATURE: Nokogumbo

Use Nokogumbo HTML parser.
2020-05-05 13:46:57 +10:00
98729c8e6e FIX: Allow embed updates of just the title 2020-04-20 14:31:24 -04:00
56a23c68f1 FIX: Embedded topics couldn't update their titles 2020-04-20 14:27:43 -04:00
b6b92a562c FEATURE: New site setting embed_unlisted (#9391)
If enabled, posts imported to discourse via embeddings will default to
unlisted until they receive a reply.
2020-04-13 15:17:02 -04:00
55b8620b43 FIX: TopicEmbed#absolutize_urls was trying to modify a frozen string 2020-03-25 12:57:54 -03:00
99fd65328c FIX: Skip absolutizing URLs when source URI is invalid 2020-02-07 10:54:24 -05:00
0fb497eb23 DEV: use Discourse.cache over Rails.cache
Discourse.cache is a more consistent method to use and offers clean fallback
if you are skipping redis

This is part of a larger change that both optimizes Discoruse.cache and omits
use of setex on $redis in favor of consistently using discourse cache

Bench does reveal that use of Rails.cache and Discourse.cache is 1.25x slower
than redis.setex / get so a re-implementation will follow prior to porting
2019-11-27 12:36:19 +11:00
55a1394342 DEV: pluck_first
Doing .pluck(:column).first is a very common pattern in Discourse and in
most cases, a limit cause isn't being added. Instead of adding a limit
clause to all these callsites, this commit adds two new methods to
ActiveRecord::Relation:

pluck_first, equivalent to limit(1).pluck(*columns).first

and pluck_first! which, like other finder methods, raises an exception
when no record is found
2019-10-21 12:08:20 +01:00
427d54b2b0 DEV: Upgrading Discourse to Zeitwerk (#8098)
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
2019-10-02 14:01:53 +10:00
d01c938e1a Revert "FIX: Use #dup instead of #+@ since content could be an instance of Nokogiri::XML::Element."
This reverts commit 50afe59306ec41a4fd74b01ca001734840adca8d.
2019-08-09 11:35:22 -03:00
50afe59306 FIX: Use #dup instead of #+@ since content could be an instance of Nokogiri::XML::Element. 2019-08-09 11:13:09 -03:00
2408d55551 FIX: embedding topics would fail with some HTML
When truncating content we try to search for first paragraph, if HTML had
no P it would fallback to first div which may have nested elements.
2019-08-07 12:45:55 +10:00
0e1d6151b9 FIX: Frozen string error in TopicEmbed.import (#7938)
When `SiteSetting.embed_truncate` is enabled (by default), the truncated
string is mutatable and does not raise an error.

However, when the setting is disabled, the `contents` string is frozen
and immutable, and will raise a `FrozenError`.
2019-07-25 09:21:01 -04:00
d369b84ced Fix modifying frozen strings error. 2019-05-13 14:31:20 +08:00
30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
6c8069c65a FIX: properly escape embed url 2019-01-07 18:11:43 +01:00
e25a6e085e FIX: drop title updates through RSS feeds
can create an update loop
2018-08-28 16:25:04 +10:00
932195d828 DEV: Update test case for TopicEmbed. 2018-08-24 09:42:12 +08:00
baf413d527 FIX: update TopicEmbed's title and user correctly 2018-08-21 18:31:01 +08:00
de92913bf4 FIX: store the topic links using the cooked upload url 2018-08-14 12:23:32 +02:00
0cc4b42180 FIX: TopicEmbed.import should update title and author 2018-05-02 17:12:31 +10:00
12706c4b29 FEATURE: support markdown rendering for embedded posts 2018-03-11 08:00:48 +05:30
5a56746610 FIX: Embedded topic was not found when URL contained query string 2018-02-14 00:28:30 +01:00
9030d3ef63 FIX: do not create duplicate topics
https://meta.discourse.org/t/duplicate-http-https-topics-are-randomly-created/77190
2018-01-04 23:53:52 +05:30
e30851e45a Move escape_uri method to a more suitable place 2017-12-12 20:17:46 +01:00
367fb1c524 FIX: Onebox fails on encoded URL.
https://meta.discourse.org/t/onebox-breaks-if-theres-chinese-text-in-url/67364
2017-09-26 18:34:54 +08:00
6f6b47f096 FIX: do not escape already escaped chars in URL 2017-09-22 17:36:44 +02:00
1a435414d5 FIX: handle URL encoded email addresses 2017-09-22 14:26:06 +02:00
b84e87dedd FIX: Don't raise an error if URL timesout. 2017-08-10 12:58:56 +09:00