By default, rails makes timestamp columns (`created_at` and
`updated_at`) non-nullable, we also have some required core and plugins
columns we wouldn't necessarily want to enforce in the intermediate DB
schema. It'll be better to set the default values for these during
import instead of enforcing these at the converter level.
This change adds support for globally modifying a column’s `nullable`
state, defaulting all `created_at` columns to be `nullable` while
allowing for table level overrides.
---------
Co-authored-by: Gerhard Schlager <gerhard.schlager@discourse.org>
* Splits the existing script into multiple classes
* Adds command for generating IntermediateDB schema (`migrations/bin/cli schema generate`)
* Changes the syntax of the IntermediateDB schema config
* Adds validation for the schema config
* It uses YAML schema aka JSON schema to validate the config file
* It generates the SQL schema file and Ruby classes for storing data in the IntermediateDB
* Updates GitHub Action for migrations
* Rubocop: Always `EnforcedShorthandSyntax` for hashes in the `migrations` directory
* Automatically load all available converter steps
* Enable YJIT at runtime, if available
* Progressbar shows skipped records and other small improvements
* Updates GitHub Actions
* Switches from `bundler/inline` to an optional group in the `Gemfile` because the previous solution didn't work well with rspec
* Adds the converter framework and tests
* Allows loading private converters (see README)
* Switches from multiple CLI tools to a single CLI
* Makes DB connections reusable and adds a new abstraction for the `IntermediateDB`
* `IntermediateDB` acts as an interface for IPC calls when a converter steps runs in parallel (forks). Only the main process writes to the DB.
* Includes a simple example implementation of a converter for now.
