* It only imports users and emails so far
* It stores mapped IDs and usernames in a SQLite DB. In the future, we might want to copy those into the Discourse DB at the end of a migration.
* The importer is split into steps which can mostly be configured with a simple DSL
* Data that needs to be shared between steps can be stored in an instance of the `SharedData` class
* Steps are automatically sorted via their defined dependencies before they are executed
* Common logic for finding unique names (username, group name) is extracted into a helper class
* If possible, steps try to avoid loading already imported data (via `mapping.ids` table)
* And steps should select the `discourse_id` instead of the `original_id` of mapped IDs via SQL
* Splits the existing script into multiple classes
* Adds command for generating IntermediateDB schema (`migrations/bin/cli schema generate`)
* Changes the syntax of the IntermediateDB schema config
* Adds validation for the schema config
* It uses YAML schema aka JSON schema to validate the config file
* It generates the SQL schema file and Ruby classes for storing data in the IntermediateDB
* Updates GitHub Action for migrations
* Rubocop: Always `EnforcedShorthandSyntax` for hashes in the `migrations` directory
* Automatically load all available converter steps
* Enable YJIT at runtime, if available
* Progressbar shows skipped records and other small improvements
* Remove unused `report_progress_in_percent` option from step
* Remove `use_custom_progress_increment` option from the step because we can figure it out by looking at the progress
* Introduce `StepTracker` to for logging warnings and errors and tracking step progress
* Make it easier to log warnings and errors in all methods of `Step` without the need to pass around a `stats` object
* DEV: Implement uploads command entrypoint
- Setup Thor UploadsCommand for CLI
- First pass at modularizing various parts of the exising `uploads_import` script
* DEV: First attempt at modularizing missing uploads fixer task
Move missing upload fix to a dedicated uploads task implementation unit
* DEV: First attempt at modularizing missing uploads uploader task
Move uploader to a dedicated uploads task implementation unit
* DEV: First attempt at modularizing missing uploads optimizer task
Move optimizer to a dedicated uploads task implementation unit
* DEV: Various follow up fixes to get optimization working
- Start threads early
- Improve "log" message formatting
- Add missing `copy_to_tempfile` method on "uploader" task
* DEV: Refactor a bit more
Deduplicate and move most of threading premitives to base task as-is
* DEV: Remove redundant condition in uploads db migration
* DEV: More deduplication
Move task retry logic to base class and tidy up other implementation
details carried over from the existing script
* Updates GitHub Actions
* Switches from `bundler/inline` to an optional group in the `Gemfile` because the previous solution didn't work well with rspec
* Adds the converter framework and tests
* Allows loading private converters (see README)
* Switches from multiple CLI tools to a single CLI
* Makes DB connections reusable and adds a new abstraction for the `IntermediateDB`
* `IntermediateDB` acts as an interface for IPC calls when a converter steps runs in parallel (forks). Only the main process writes to the DB.
* Includes a simple example implementation of a converter for now.
* Moves existing files around. All essential scripts are in `migrations/bin`, and non-essential scripts like benchmarks are in `migrations/scripts`
* Dependabot configuration for migrations-tooling (disabled for now)
* Updates test configuration for migrations-tooling
* Shorter configuration for intermediate DB for now. We will add the rest table by table.
* Adds a couple of benchmark scripts
* RSpec setup especially for migrations-tooling and the first tests
* Adds sorting/formatting to the `generate_schema` script
