Commit Graph

683 Commits

Author SHA1 Message Date
d7bd69583f fix vanilla bulk importer resume categories and private topics 2018-03-01 15:29:38 -05:00
ae9b1e8554 FEATURE: IP.Board 3 importer 2018-03-01 09:43:32 +01:00
e997cc7b77 add more image extensions to downsize_uploads.rb 2018-02-27 22:05:35 -05:00
931caa4335 add a script that can downsize uploads 2018-02-27 16:23:48 -05:00
82e68670bf FIX: SimplePress importer wasn't handling increment imports properly 2018-02-24 00:42:17 +01:00
Sam
720e1965e3 FEATURE: add category suppress from latest
In the past we used suppress_from_homepage, it had mixed semantics
it would remove from category list if category list was on home and
unconditionally remove from latest.

New setting explicitly only removes from latest list but leaves the
category list alond
2018-02-22 09:56:35 +11:00
Sam
73a492f721 minor changes to discourse bench
Ruby master is not compatible with bootsnap atm
2018-02-20 14:41:21 +11:00
Sam
a3c7ee09b6 FIX: ruby bench not working properly
- Remove thin which is no longer supported
- Bypass admin api rate limiting in profile environment
- Admin password was too short
- Run by default in concurrency 1 mode
- A skip bundle assets flag to speed up local testing
2018-02-19 11:37:16 +11:00
Sam
5faffeeb82 demo script for profiling middleware stack 2018-02-15 14:36:40 +11:00
Sam
c89b42c488 PERF: only require the rss library if used
Before:

Total allocated: 257909321 bytes (2514134 objects)
Total retained:  39681579 bytes (343387 objects)

allocated memory by gem
-----------------------------------
  42875979  rss

retained memory by gem
-----------------------------------
   2080188  rss

retained objects by gem
-----------------------------------
     13052  rss

After:

Total allocated: 210562047 bytes (2252030 objects)
Total retained:  37433816 bytes (328635 objects)

----

So, 2 less megabytes on boot and 13000 objects stuck in ruby heaps forever.
2018-02-15 13:11:33 +11:00
479f7ed18f Ignore case when removing mailing list name from subject 2018-02-12 21:41:58 +01:00
31f5ca6180 improve simple press importer 2018-02-07 17:37:51 +01:00
0edd386b48 FEATURE: Vanilla bulk importer 2018-02-02 16:28:51 -05:00
25ec077eca rename 'min_private_message_{post/title}_length' to 'min_personal_message_{post/title}_length' 2018-02-01 13:25:29 +05:30
421713f1be include saved-searches in plugin-translations.rb 2018-01-31 11:19:21 -05:00
cc830f7119 sorry rubocop 2018-01-30 14:37:15 -05:00
db17c531d6 Feature: Invision Power Board Importer 2018-01-30 08:44:02 -08:00
192a0886e2 FIX: BBCode to Markdown conversion in phpBB3 importer was broken
This fixes the conversion for quotes, code blocks and lists (except for nested lists). It also discourages the usage of the ruby-bbcode-to-md gem.
2018-01-30 12:50:42 +01:00
c26db2116c FIX: phpBB3 importer imported some users as anonymous users 2018-01-30 12:50:42 +01:00
a224459960 bulk importer shouldn't try to update primary key sequences to -1 2018-01-19 15:01:00 -05:00
db287ad58e FIX: invalid SQL in vanilla_mysql import script 2018-01-19 11:53:07 -05:00
6500343431 FIX: mbox importer didn't detected already indexed files 2018-01-17 17:03:53 +01:00
bb54eb1192 Improvements to mbox importer
* store time it took to index message in DB (to find performance issues)
* ignore listserv specific files
* better examples for split_regex
* first email in mbox shouldn't contain the split string
* always lock the DB in exclusive mode
* save email within transaction
* messages can be grouped by subject and use original order (for Listserv)
* adds option to index emails without running the import
2018-01-17 12:04:57 +01:00
Sam
fc05164667 demo script for demonizing using fork exec
minor refinements to demon
2018-01-11 13:51:52 +11:00
86d13c0d62 Pull translations for discourse-characters-required plugin 2018-01-05 14:35:52 +01:00
77a92e8878 Allow user staging via setting (#5468) 2018-01-04 09:17:35 +01:00
3cba858930 add a script to update plugin translations from Transifex 2017-12-29 16:57:06 -05:00
a93bf186c5 FIX: validation of website failed in base importer 2017-12-28 14:51:43 +01:00
cc46ab5bb1 smf2: deal with duplicate category names (#5435) 2017-12-27 16:56:36 +11:00
081584c676 Allow running smoke tests within docker image with latest code 2017-12-20 15:59:21 +01:00
cafe69caac Refactor mbox import script 2017-12-13 22:03:31 +01:00
3190c13c22 import staged users as inactive in mbox import 2017-12-13 08:45:43 +05:30
19ee2c3fac Add presence plugin to pull translations script to fix the build 2017-12-11 20:37:54 +01:00
7c3a087ef5 FIX: Script for pulling translations ignored exit code 2017-12-06 17:05:10 +01:00
16738cfb1b FEATURE: convert plain text emails to markdown 2017-12-06 01:47:51 +01:00
ef0f6f5b97 Merge pull request #5308 from bill2004158/master
fix error "extension "hstore" already exists"
2017-12-05 11:22:13 +08:00
27d94b3a0b Remove unused option from discourse script 2017-11-23 15:22:00 +01:00
32dd1e66be improvements to the mbox import script
* ignores dot-files and empty emails
* new setting to prefer HTML over plaintext emails during import
* restore original site settings at the end of import
* elided content of HTML mails was not put inside details block
2017-11-18 17:16:44 +01:00
c90c4b9703 sadly I can't remove that locale_file_walker dependency from pull_translations.rb 2017-11-16 15:05:45 -05:00
94ceb06b02 fix user lookup by email in disqus import script 2017-11-16 16:26:18 +01:00
0cdded8079 Merge pull request #5303 from vinothkannans/export-multiple-categories
FEATURE: Option to export multiple categories using export_category method
2017-11-15 16:07:21 -05:00
06a6ddc3ba handle plaintext and HTML emails in mbox importer 2017-11-15 20:22:11 +01:00
ad0bf66c33 fix error "extension "hstore" already exists"
Error executing 'postInstallation': ERROR:  extension "hstore" already exists
2017-11-13 10:31:31 +08:00
d9823f69c6 FEATURE: Option to export multiple categories using export_category method 2017-11-11 19:11:16 +05:30
5d8508c523 New yahoo groups importer 2017-11-09 11:07:57 +01:00
667b025d12 make rubocop happy 2017-11-07 14:20:46 -05:00
4f0bdec370 some improvements for importers (#5295)
* decode html entities within code blocks

* Only import users that actually participated in the bbpress part of Wordpress; import password hashes

* create permalinks for topics

* Better handling of [code] blocks
2017-11-07 17:50:43 +01:00
7f33f7850a Merge pull request #5270 from vinothkannans/rake_categories
FEATURE: Rake task to export and import category structure
2017-11-01 21:36:03 +05:30
a00af4d85a FEATURE: Rake task to export and import category structure 2017-11-01 17:17:05 +05:30
9cbb90c5ed add importer for modx forum (#5239) 2017-10-31 10:50:34 +11:00