Commit Graph

4 Commits

Author SHA1 Message Date
251cac39af DEV: Adds a basic importer for the IntermediateDB
* It only imports users and emails so far
* It stores mapped IDs and usernames in a SQLite DB. In the future, we might want to copy those into the Discourse DB at the end of a migration.
* The importer is split into steps which can mostly be configured with a simple DSL
* Data that needs to be shared between steps can be stored in an instance of the `SharedData` class
* Steps are automatically sorted via their defined dependencies before they are executed
* Common logic for finding unique names (username, group name) is extracted into a helper class
* If possible, steps try to avoid loading already imported data (via `mapping.ids` table)
* And steps should select the `discourse_id` instead of the `original_id` of mapped IDs via SQL
2025-04-07 17:22:36 +02:00
71a90dcba2 DEV: Refactor migrations-tooling
* Updates GitHub Action for migrations
* Rubocop: Always `EnforcedShorthandSyntax` for hashes in the `migrations` directory
* Automatically load all available converter steps
* Enable YJIT at runtime, if available
* Progressbar shows skipped records and other small improvements
2025-04-07 17:22:36 +02:00
a48af2f120 DEV: Store details log entries in converter as JSON (#29778)
Plus small DB related fixes
2024-11-19 23:54:00 +01:00
7c3a29c9d6 DEV: Add converter framework for migrations-tooling (#28540)
* Updates GitHub Actions
* Switches from `bundler/inline` to an optional group in the `Gemfile` because the previous solution didn't work well with rspec
* Adds the converter framework and tests
* Allows loading private converters (see README)
* Switches from multiple CLI tools to a single CLI
* Makes DB connections reusable and adds a new abstraction for the `IntermediateDB`
* `IntermediateDB` acts as an interface for IPC calls when a converter steps runs in parallel (forks). Only the main process writes to the DB.
* Includes a simple example implementation of a converter for now.
2024-09-09 17:14:39 +02:00