Commit Graph

37 Commits

Author SHA1 Message Date
30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
24347ace10 FIX: Properly associate user_profiles background urls via upload id.
`Upload#url` is more likely and can change from time to time. When it
does changes, we don't want to have to look through multiple tables to
ensure that the URLs are all up to date. Instead, we simply associate
uploads properly to `UserProfile` so that it does not have to replicate
the URLs in the table.
2019-05-02 14:58:24 +08:00
afa22a0c6f REFACTOR: more 'fake_email' to base importer 2018-10-22 11:12:40 +02:00
8b20e2500a Remove unnecessary line 2018-10-19 15:48:48 +02:00
637123ff6f Merge users based on their email in vBulletin importer 2018-10-19 15:16:45 +02:00
53aa0344bf FIX: properly import vBulletin's hashed password 2018-10-18 10:22:55 +02:00
Sam
5f64fd0a21 DEV: remove exec_sql and replace with mini_sql
Introduce new patterns for direct sql that are safe and fast.

MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API

- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder

See more at: https://github.com/discourse/mini_sql
2018-06-19 16:13:36 +10:00
fc973f9363 Improve the VBulletin importer (#5922) 2018-06-12 20:41:21 +02:00
5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
9641d2413d REFACTOR: upload workflow creation into UploadCreator
- Automatically convert large-ish PNG/BMP to JPEG
- Updated fast_image to latest version
2017-05-11 00:16:57 +02:00
0d250c3935 50% faster vBulletin 4 importer 2017-02-01 14:33:09 +01:00
3a5c0c5605 add env variables for vBulletin import script 2017-01-31 10:31:18 +05:30
bc621a704e fix close topics vb4 importer 2016-12-15 14:20:05 +02:00
59523aef9d more improvements to vBulletin import script 2016-12-05 17:44:18 +05:30
f68194cf8e fix nil for vb4 importer 2016-12-05 04:16:59 +02:00
52749c0121 imporoving vb4 importer 2016-12-04 00:02:47 +02:00
92e716a1fd fix vbulletin import script 2016-09-14 08:15:48 +05:30
201d344a2d improve vBulletin import script 2016-07-10 14:51:31 +05:30
6238a43f93 Fix base and vBulletin importer 2015-10-21 19:07:31 +02:00
821124f3fd FEATURE: Skip batches if all records exist
Update all import scripts to take advantage of all_records_exist?
2015-09-21 17:13:02 -07:00
7737cc2828 vBulletin: ability to import attachments from filesystem 2015-05-15 19:39:15 +08:00
a412e9bede Fix gem load order for all importers
https://meta.discourse.org/t/migrating-from-mybb/25563/8
2015-03-18 20:31:02 +01:00
b5426763e4 FIX: Importers should allow categories with existing name if the parent category is different
This changes the content of `@categories_lookup` from `Category` objects
to IDs since the category names aren't needed anymore. The lookup
method has been renamed too.
2015-03-12 21:15:02 +01:00
c35cdfcf6b Update vbulletin.rb 2015-03-10 00:31:12 +10:30
31340de446 don't break the vBulletin importer when PrettyText times out 2015-01-26 20:35:30 +01:00
eecc573fbc FIX: don't break import when raw can't be preprocessed (vBulletin importer) 2015-01-21 09:36:46 +01:00
6c4d852011 Improve vBulletin importer
- FEATURE: TopicCreator now supports 'pinned_at' parameter
- FIX: 🐛 FIX TopicQuerySQL to support pinned topic older than 2010
- FIX: 🐛 Properly remove all HTML Entities from Usernames/Titles/Category Names/Groups in vBulletin importer
- FIX: 🐛 Properly handle specific vBulletin BBCode (quotes/mentions)
- FIX: 🐛 Make sure we generate a username from the name of the user instead of a fake email
- FEATURE: Allow for custom timezone in vBulletin importer
- FEATURE: Support for profile pictures/background in vBulletin importer
- FIX: 🐛 merge the categories tree to only 2 levels in vBulletin importer
2015-01-19 15:00:55 +01:00
5045d8677c new vBulletin importer (uses mysql dump instead of multiple csv files) 2014-12-22 13:22:16 +01:00
d1608bdfe9 FIX: better error handling in vBulletin importer 2014-09-08 11:02:54 +02:00
4907053cc4 FIX: vBulletin pre-processing regexes order 2014-09-05 18:44:34 +02:00
85cbb001ae FIX: properly close topics in vBulletin importer
FEATURE: add backtrace when an exception happen (importers)
FIX: post-processing should also happen on first posts (vBulletin
importer)
PERF: faster topic bypass when already imported
2014-09-04 17:55:05 +02:00
e823f568a7 FIX: preprocessing regexes in vBulletin importer 2014-08-29 13:11:56 +02:00
9b86d1507d VBULLETIN: add some stats about the imported groups 2014-08-25 12:33:08 +02:00
7aaf718cf3 update bbcode dialect and fix vBulletin importer 2014-08-25 10:48:29 +02:00
841d547a48 FIX: vBulletin wasn't importing posters... 2014-08-20 23:04:43 +02:00
66aaa9329e FEATURE: vBulletin importer 2014-08-18 13:04:08 +02:00
fab2c8f816 Create vbulletin.rb 2014-07-09 22:44:06 +02:00