Commit Graph

409 Commits

Author SHA1 Message Date
9b651adadb FIX: mbox importer should ignore emails without date 2018-03-13 13:42:57 +01:00
Sam
ddcca1d5cc lint 2018-03-08 15:16:12 +11:00
4b8ff7a057 Update xenforo.rb (#5622)
#Fix for the error: xenforo.rb: 160: in `gsub!': invalid byte sequence in UTF-8 (ArgumentError)
2018-03-07 22:57:38 -05:00
07bc52dc69 import topic views in phpbb3 2018-03-07 19:27:03 +01:00
dc32ee5cbf Improvements to mbox import script
* Ignore errors during indexing and show information about the message causing the problem
* Always activate imported users if they aren't staged
2018-03-06 11:32:12 +01:00
49400337b6 Improve IPBoard3 importer
- fix encoding issues
- close topics
- don't import queued & not approved topics
- don't import deleted posts
2018-03-05 10:24:42 +01:00
ae9b1e8554 FEATURE: IP.Board 3 importer 2018-03-01 09:43:32 +01:00
82e68670bf FIX: SimplePress importer wasn't handling increment imports properly 2018-02-24 00:42:17 +01:00
Sam
720e1965e3 FEATURE: add category suppress from latest
In the past we used suppress_from_homepage, it had mixed semantics
it would remove from category list if category list was on home and
unconditionally remove from latest.

New setting explicitly only removes from latest list but leaves the
category list alond
2018-02-22 09:56:35 +11:00
479f7ed18f Ignore case when removing mailing list name from subject 2018-02-12 21:41:58 +01:00
31f5ca6180 improve simple press importer 2018-02-07 17:37:51 +01:00
25ec077eca rename 'min_private_message_{post/title}_length' to 'min_personal_message_{post/title}_length' 2018-02-01 13:25:29 +05:30
cc830f7119 sorry rubocop 2018-01-30 14:37:15 -05:00
db17c531d6 Feature: Invision Power Board Importer 2018-01-30 08:44:02 -08:00
192a0886e2 FIX: BBCode to Markdown conversion in phpBB3 importer was broken
This fixes the conversion for quotes, code blocks and lists (except for nested lists). It also discourages the usage of the ruby-bbcode-to-md gem.
2018-01-30 12:50:42 +01:00
c26db2116c FIX: phpBB3 importer imported some users as anonymous users 2018-01-30 12:50:42 +01:00
db287ad58e FIX: invalid SQL in vanilla_mysql import script 2018-01-19 11:53:07 -05:00
6500343431 FIX: mbox importer didn't detected already indexed files 2018-01-17 17:03:53 +01:00
bb54eb1192 Improvements to mbox importer
* store time it took to index message in DB (to find performance issues)
* ignore listserv specific files
* better examples for split_regex
* first email in mbox shouldn't contain the split string
* always lock the DB in exclusive mode
* save email within transaction
* messages can be grouped by subject and use original order (for Listserv)
* adds option to index emails without running the import
2018-01-17 12:04:57 +01:00
77a92e8878 Allow user staging via setting (#5468) 2018-01-04 09:17:35 +01:00
a93bf186c5 FIX: validation of website failed in base importer 2017-12-28 14:51:43 +01:00
cc46ab5bb1 smf2: deal with duplicate category names (#5435) 2017-12-27 16:56:36 +11:00
cafe69caac Refactor mbox import script 2017-12-13 22:03:31 +01:00
3190c13c22 import staged users as inactive in mbox import 2017-12-13 08:45:43 +05:30
16738cfb1b FEATURE: convert plain text emails to markdown 2017-12-06 01:47:51 +01:00
32dd1e66be improvements to the mbox import script
* ignores dot-files and empty emails
* new setting to prefer HTML over plaintext emails during import
* restore original site settings at the end of import
* elided content of HTML mails was not put inside details block
2017-11-18 17:16:44 +01:00
94ceb06b02 fix user lookup by email in disqus import script 2017-11-16 16:26:18 +01:00
06a6ddc3ba handle plaintext and HTML emails in mbox importer 2017-11-15 20:22:11 +01:00
5d8508c523 New yahoo groups importer 2017-11-09 11:07:57 +01:00
667b025d12 make rubocop happy 2017-11-07 14:20:46 -05:00
4f0bdec370 some improvements for importers (#5295)
* decode html entities within code blocks

* Only import users that actually participated in the bbpress part of Wordpress; import password hashes

* create permalinks for topics

* Better handling of [code] blocks
2017-11-07 17:50:43 +01:00
9cbb90c5ed add importer for modx forum (#5239) 2017-10-31 10:50:34 +11:00
aec5de451b FIX: compare imported ids as string instead of converting to integer, same as LookupContainer 2017-10-23 14:01:57 -04:00
0d257058c9 make rubocop 🤖 happy 2017-10-23 10:21:50 +02:00
20e7e285d6 add users with invalid email addresses (#5224)
* add users with invalid email addresses

* start to add suspend user stff

* don't create a suspend_user function

* include reason for suspension
2017-10-23 10:18:44 +02:00
6c829c24d7 escaping the subject isn't needed in the mbox importer 2017-10-19 15:25:20 +02:00
c41880ab19 Improvements to the experimental mbox importer
* Disable journaling to improve performance in Docker
* Use the email cooking method
* Store IncomingEmail in order find related posts by Message-ID
* Escape HTML in imported messages
2017-10-19 14:27:40 +02:00
e9fb2c5756 phpBB3 importer: Fix validation of site settings 2017-08-29 01:48:30 +02:00
3f8e535692 FEATURE: bbpress importer can import private messages 2017-08-21 16:39:57 -04:00
5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
32ef30cff9 phpBB3 importer: Permalink for first post of each topic was missing 2017-07-25 23:29:56 +02:00
5be52fa821 phpBB3 importer: Use permalink normalization for category links 2017-07-25 21:14:46 +02:00
ed5d322196 FIX: handle existing users in imports 2017-07-24 16:22:56 +05:30
d38b55e0c8 Merge pull request #4959 from farmdawgnation/minor-mbox-import-fix
Add a nil check on the connection before attempting to exec it
2017-07-21 09:45:15 +09:00
d0b027d88d FEATURE: phase 1 of supporting multiple email addresses 2017-07-20 11:22:27 +09:00
d0c5205a52 Feature: Change markdown engine to markdown it
This commit removes the old evilstreak markdownjs engine.

- Adds specs to WhiteLister and changes it to stop using globals
    (Fixes large memory leak)
- Fixes edge cases around bbcode handling
- Removes mdtest which is no longer valid (to be replaced with
    CommonMark)
- Updates MiniRacer to correct minor unmanaged memory leak
- Fixes plugin specs
2017-07-17 11:41:34 -04:00
Sam
79a084dd58 Revert "remove old markdown engine work-in-progress"
This reverts commit ee470b531788b71c22721562e8bbb846004a9bc7.
2017-07-12 18:10:51 -04:00
ee470b5317 remove old markdown engine work-in-progress 2017-07-12 17:44:40 -04:00
5b11391588 Add a nil check on the connection before attempting to exec it 2017-07-07 11:11:43 -04:00
1c9cab0908 mybb: disable outgoing mail 2017-06-30 16:28:14 -07:00