Commit Graph

1223 Commits

Author SHA1 Message Date
e8bf304f05 FIX: Add popper sourcemap 2020-07-15 15:29:12 -04:00
339549d14a Support plugin and Theme compatibility version manifests (#9995)
Adds a new rake task `plugin:checkout_compatible_all` and
`plugin:checkout_compatible[plugin-name]` that check out compatible plugin
versions.

Supports a .discourse-compatibility file in the root of plugins and themes that
list out a plugin's compatibility with certain discourse versions:

eg: .discourse-compatibility
```
2.5.0.beta6: some-git-hash
2.4.4.beta4: some-git-tag
2.2.0: git-reference
```

This ensures older Discourse installs are able to find and install older
versions of plugins without intervention, through the manifest only.

It iterates through the versions in descending order. If the current Discourse
version matches an item in the manifest, it checks out the listed plugin target.
If the Discourse version is greater than an item in the manifest, it checks out
the next highest version listed in the manifest.

If no versions match, it makes no change.
2020-07-06 14:48:00 -07:00
69803599a9 DEV: Refactor seed data filter
Added a small helper class to for seed data because we need to add the
same filter to multisite:migrate as we have in db:migrate. Having this
filter in both places means we can get rid of the SKIP_SEED flag.
2020-06-26 14:36:50 -06:00
c16ad39f8e DEV: Run seeds irregardless of post deploy migration flag.
Follow up to 01937b2d
2020-06-26 11:04:34 +08:00
01937b2de2 Revert "FIX: Seed needs to run before optimizing site icons."
This reverts commit 715ddf38618555c7ba798f8526f85f79a7a5d365.
2020-06-26 11:03:47 +08:00
715ddf3861 FIX: Seed needs to run before optimizing site icons. 2020-06-26 08:58:53 +08:00
01b6349a67 DEV: Add skip seed flag (#10116)
* add a flag to skip seed

* only seed when running post deployment migrations
2020-06-25 10:14:58 -06:00
0384b6d910 FIX: multisite:migrate failing to properly seed data. 2020-06-23 09:10:02 +08:00
2e1efbde52 FIX: Pass local scope variable to inner function 2020-06-22 19:23:59 +03:00
e29afa200a FIX: Cleanup migrations with timestamps in the future
A future-dated migration was accidently introduced by me in 45c399f0. This was removed in b9762afc, but other migrations had already been generated based on its incorrect date. This commit removes the offending data in the schema_migrations table, and corrects the version in the published_pages migration.

This commit also adds a check to db:migrate which raises an error when invalid migration timestamps are used.
2020-06-17 15:58:22 +01:00
45eb97c202 FIX: Thread safety issues with multisite:migrate and SeedFu. 2020-06-17 16:15:43 +08:00
669c940ec3 Revert "DEV: Remove the remaining ENV["TRAVIS"] usage (#10041)"
This reverts commit 78aff841e3215c875e79692fbeaf8d18c4514006.

See https://review.discourse.org/t/dev-remove-the-remaining-env-travis-usage-10041/12737/4?u=cvx
2020-06-16 19:42:00 +02:00
78aff841e3 DEV: Remove the remaining ENV["TRAVIS"] usage (#10041) 2020-06-16 17:41:15 +02:00
a2713578dd DEV: Allow plugins to exclude seed data
This allows plugins to specify if they would like to filter out any seed
data files from running during migrations.
2020-06-15 15:30:25 -06:00
5f3fd23fa8 DEV: Add stub flush_sw task to prevent rebuild errors (#10021) 2020-06-10 13:07:37 -07:00
d4caf69ed7 DEV: makes SKIP_INSTALL_PLUGINS called last and global (#9990) 2020-06-05 17:59:23 +02:00
1c48853ede DEV: allow skipping of a single plugin when installing
SKIP_INSTALL_PLUGINS can be used to tell plugin:install_all_official
to skip a plugin.

Comma seperated list
2020-06-05 18:26:25 +10:00
76af25f753 FEATURE: Support append when bulk tagging via rake (#9978)
New argument option for the `tags:bulk_tag_category` rake task.
Backwards compatible - defaults to false (no append) if no argument given.
2020-06-04 09:33:48 -04:00
07774894a6 DEV: notify when the rake ask is completed 2020-06-04 12:00:06 +05:30
a93d24501c FIX: base import script was not updating first_post_created_at column
FEATURE: new rake task to update first_post_created_at column

The not-equal operator (`<>`) in PostgreSQL does not compare values
with NULL. We should instead use `IS DISTINCT FROM` when comparing
values with NULL.
2020-06-04 11:26:40 +05:30
81e6bc7a0f FEATURE: Add uploads:batch_migrate_from_s3 task to limit total posts migrated at once (#9933)
Allow limiting the number of migrations to do at once, both to do migrations that
have impact limited to multiple off-peak usage hours to reduce user impact from
a migration, and to allow tests that do only a very small number for test
purposes. ("Give me a ping, Vasili. One ping only, please.")
2020-06-04 09:48:11 +10:00
f683c5d0e0 DEV: Check English locale for errors in CI
Moves the most important checks into a linter. It gets executed by Lefthook as well as the docker rake task and Github actions. Doing those checks in rspec takes too long and it produces errors when the discourse:test Docker image contains old, invalid locale files.
2020-06-03 21:54:58 +02:00
57a3d4e0d2 FEATURE: whitelist theme repo mode (experimental)
In some restricted setups all JS payloads need tight control.

This setting bans admins from making changes to JS on the site and
requires all themes be whitelisted to be used.

There are edge cases we still need to work through in this mode
hence this is still not supported in production and experimental.

Use an example like this to enable:

`DISCOURSE_WHITELISTED_THEME_REPOS="https://repo.com/repo.git,https://repo.com/repo2.git"`

By default this feature is not enabled and no changes are made.

One exception is that default theme id was missing a security check
this was added for correctness.
2020-06-03 13:19:57 +10:00
6358e79f0f FIX: catch NoMethodError during precompile_css too (#9940)
Ran into this on heroku:

    NoMethodError: undefined method `enabled?' for #<Theme:0x0000558f069e7718>
    vendor/bundle/ruby/2.6.0/gems/activemodel-6.0.1/lib/active_model/attribute_methods.rb:431:in `method_missing'
    app/models/theme.rb:155:in `block (2 levels) in transform_ids'
    vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.1/lib/active_record/relation/delegation.rb:85:in `each'
    vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.1/lib/active_record/relation/delegation.rb:85:in `each'
    vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.1/lib/active_record/relation/query_methods.rb:260:in `select'
    vendor/bundle/ruby/2.6.0/gems/activerecord-6.0.1/lib/active_record/relation/query_methods.rb:260:in `select'
    app/models/theme.rb:155:in `block in transform_ids'
    app/models/theme.rb:105:in `get_set_cache'
    app/models/theme.rb:140:in `transform_ids'
    app/models/theme.rb:321:in `list_baked_fields'
    app/models/theme.rb:317:in `resolve_baked_field'
    lib/stylesheet/manager.rb:289:in `theme_digest'
2020-06-02 15:18:03 +10:00
7635c18a14 DEV: ensures highlightjs is correctly tested (#9923) 2020-05-29 13:05:44 +02:00
d9a02d1336 Revert "Revert "Merge branch 'master' of https://github.com/discourse/discourse""
This reverts commit 20780a1eeed56b321daf18ee6bbfe681a51d1bf4.

* SECURITY: re-adds accidentally reverted commit:
  03d26cd6: ensure embed_url contains valid http(s) uri
* when the merge commit e62a85cf was reverted, git chose the 2660c2e2 parent to land on
  instead of the 03d26cd6 parent (which contains security fixes)
2020-05-23 00:56:13 -04:00
20780a1eee Revert "Merge branch 'master' of https://github.com/discourse/discourse"
This reverts commit e62a85cf6fd81a2a34aff6144bd36b9ac459964a, reversing
changes made to 2660c2e21d84bea667e1ea339f91cda352328062.
2020-05-22 20:25:56 -07:00
779dc30d2e FIX: Join bookmarks migration on users to avoid missing user records
* the post_actions table has no FK to users, so if a user has been
  deleted we may end up with dangling post_action records, which then
  interferes with the bookmarks migration because bookmarks DO have
  an FK to users
2020-05-19 16:31:05 +10:00
869f9b20a2 PERF: Dematerialize topic_reply_count (#9769)
* PERF: Dematerialize topic_reply_count

It's only ever used for trust level promotions that run daily, or compared to 0. We don't need to track it on every post creation.

* UX: Add symbol in TL3 report if topic reply count is capped

* DEV: Drop user_stats.topic_reply_count column
2020-05-14 15:42:00 -07:00
42b300b9be DEV: Skip I18n in plugins until they can be updated 2020-05-14 10:40:53 -04:00
1c0c425f38 DEV: Detect correct data dir using ENV 2020-05-12 13:10:57 -03:00
8311374a5a DEV: Replace version.js.erb with pre generated file 2020-05-11 15:43:09 -04:00
d2b16a7618 DEV: Generate emoji/data on demand (#9744)
These emoji rarely change and can be re-generated when we make changes
to them.
2020-05-11 15:29:46 -04:00
34e5f807ec DEV: User UPPER_CASE for constants 2020-05-08 15:50:55 -04:00
7f373e8b93 DEV: Don't use js.erb for constants
Adds a new rake task to auto generate a constants.js file with the
constants present. This makes migrating to Ember CLI easier, but also
slightly speeds up asset compilation by having to do less work.

If the constants change you need to run:
`rake javascripts:update_constants`
2020-05-08 14:14:01 -04:00
72ad701df0 DEV: Stub #flush in StdOutDemux for multisite:migrate
https://meta.discourse.org/t/multisite-migrate-error/150579/2
2020-05-06 11:58:35 +01:00
9bff0882c3 FEATURE: Nokogumbo (#9577)
* FEATURE: Nokogumbo

Use Nokogumbo HTML parser.
2020-05-05 13:46:57 +10:00
08fbf199ad FIX: S3 rake task can ignore yarn.lock 2020-04-29 13:14:04 -04:00
9ec908950d DEV: Better error handling for s3 task 2020-04-29 12:54:39 -04:00
ec2943c5bc DEV: Update jquery.fileupload and dependencies (#9466) 2020-04-28 10:39:29 -04:00
6b62d75c50 FIX: remove unsuppored rake task
rebuilding user_actions is not something that should be done.

Plugins such as solved and assigned extend it, there are tons of
little rules that were not captured in `user_actions:rebuild`
2020-04-28 16:27:42 +10:00
b810fc282d FEATURE: Digital Ocean Spaces compatibility for Object Storage
DO does not implement tagging support for S3 objects. Removing our default
empty tag fixes compatibility.

The expire_missing_assets rake task can't be used with that service still,
but this patch allows normal operation.
2020-04-22 13:57:14 -03:00
628ba9d1e2 FEATURE: Promote bookmarks with reminders to core functionality (#9369)
The main thrust of this PR is to take all the conditional checks based on the `enable_bookmarks_with_reminders` away and only keep the code from the `true` path, making bookmarks with reminders the core bookmarks feature. There is also a migration to create `Bookmark` records out of `PostAction` bookmarks for a site.

### Summary

* Remove logic based on whether enable_bookmarks_with_reminders is true. This site setting is now obsolete, the old bookmark functionality is being removed. Retain the setting and set the value to `true` in a migration.
* Use the code from the rake task to create a database migration that creates bookmarks from post actions.
* Change the bookmark report to read from the new table.
* Get rid of old endpoints for bookmarks
* Link to the new bookmarks list from the user summary page
2020-04-22 13:44:19 +10:00
e41f1e6ec2 FEATURE: Rake task to export groups (#9450) 2020-04-17 14:59:54 -07:00
51672b9121 FIX: Minor bookmark with reminder issue cleanup (#9436)
* Count user summary bookmarks from new Bookmark table if bookmarks with reminders enabled
* Update topic user bookmarked column when new topic bookmark changed
* Make in:bookmarks search work with new bookmarks
* Fix batch inserts for bookmark rake task (and thus migration). We were only inserting one bookmark at a time, completely defeating the purpose of batching!
2020-04-16 11:32:21 +10:00
f07c4a781c DEV: Add options to theme install rake task - more options (#9394) 2020-04-14 09:17:00 -05:00
e2284cf739 Revert "We have had errors reported due to migrations breaking and are reverting"
This reverts commit 8b46f14744d4bab9339d075825914d86da6bd5eb.

It corrects the reason for the revert:

We rely on SafeMigrate existing cause we call it from migrations,
Zeitwerk will autoload it.

Instead of previous pattern we explicitly bypass all the hacks in
production mode.

We need to disable SafeMigrate cause it is not thread safe.

A thread safe implementation is possible but not worth the effort,
we catch the issues in dev and test.
2020-04-14 11:31:07 +10:00
8b46f14744 We have had errors reported due to migrations breaking and are reverting
these series of commits.

See:
https://meta.discourse.org/t/new-installation-fails-on-migration/147425

This reverts commit 80e832662b719425872e7ac3736fb9fc56d85aca.
This reverts commit de5f2d33087c8e9c12f83d573015d275098bd68f.
This reverts commit 6b192d29fa095e91cb40b71e76dfc811f4aa205c.
This reverts commit a5b582f686c0a8427aad3f6caf9f3a8105e66ee9.
This reverts commit 708dd97dfd0380c703678a0ee070dc62880e57f1.
2020-04-09 14:18:56 -04:00
80e832662b DEV: fake stdout must respond to close
If it does not respond to close Logger will refuse to use it
2020-04-09 14:36:01 +10:00
a5b582f686 DEV: demux stdout when running multisite migrate
This avoids mixing up output in such a way that we can not tell which
site ran which migrations

Avoids threads all fighting for output
2020-04-09 12:50:14 +10:00