Skip to content

Keep pgroll state schema in sync with pgroll binary version #876

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jun 12, 2025

Conversation

andrew-farries
Copy link
Collaborator

Background

pgroll stores its state in the pgroll schema in the target database. This schema includes the data and SQL functions that it needs to operate. Including:

  • the pgroll.migrations table: tracks all schema changes applied to all schema in the target database.
  • various SQL functions like:
    • is_active_migration_period - determines if there is an in-progress migration.
    • read_schema - returns all tables, constraints, indexes etc in the given schema.
    • raw_migration - installed by the event trigger to capture inferred migrations.

The pgroll schema is created when a user runs pgroll init. Internally this runs init.sql against the target database. init.sql is written to be idempotent; a user can run pgroll init multiple times without fear of corrupting pgroll's internal state.

When a user upgrades their version of pgroll , there is no automated mechanism to ensure that pgroll init is re-run. This means that the new version of pgroll is not in sync with any changes to pgroll's internal state tables/functions required by that new version.


This PR keeps the version of pgroll and its internal pgroll schema in sync:

The pgroll_version table

pgroll will track the version number of the pgroll binary used to initialize pgroll's internal state in a new pgroll_version table:

+-------------+-------------------------------+
| version     | initialized_at                |
|-------------+-------------------------------|
| 0.14.0      | 2025-06-09 14:04:23.601366+00 |
+-------------+-------------------------------+

The table will contain exactly one row. The table is created by init.sql as follows:

-- Table to track pgroll binary version
CREATE TABLE IF NOT EXISTS placeholder.pgroll_version (
    version text NOT NULL,
    initialized_at timestamptz NOT NULL DEFAULT CURRENT_TIMESTAMP,
    PRIMARY KEY (version)
);

When the user runs pgroll init the pgroll_version table is truncated and the current version of the pgroll binary is inserted into the table.

Automatic upgrades

When a user upgrades their version of pgroll , pgroll will automatically re-initialize its internal state to ensure that there is no version skew between the pgroll binary and the version of pgroll's internal state.

For example, a user has pgroll 0.14.0 installed. The pgroll_version table looks ike this

+-------------+-------------------------------+
| version     | initialized_at                |
|-------------+-------------------------------|
| 0.14.0      | 2025-06-09 14:04:23.601366+00 |
+-------------+-------------------------------+

The user installs pgroll 0.15.0. The first time the user runs a pgroll command that requires a connection to the target database (technically whenever a state.State instance is constructed), the version stored in the pgroll_version table is checked against the current pgroll binary version. If the state version is less than the binary version (using a standard semver comparison), then the state is re-initialized and the pgroll_version table is updated.

Disallow downgrades

In the case where a user attempts to use a version of pgroll that is older than the version stored in the pgroll_version table, the pgroll command will display an error and ask the user to upgrade their version of pgroll.

Using an older version of pgroll against newer internal state is not supported and a downgrade risks permanent data loss so aborting is the correct option here.

Edge cases

Missing pgroll_version table

For versions of pgroll's internal state that don’t have a pgroll_version table, the version comparison will always report that the internal state version is smaller than the current pgroll binary version. This will cause re-initialization and creation of the pgroll_version table.

Development versions

The version number is injected into the pgroll CLI using linker flags at build time during our CI process. For local builds that don’t perform this version injection, the version is set to the string “development”.

  • A development build of pgroll will never cause re-initialization of pgroll internal state.
  • No version of pgroll will re-initialize If the version in the pgroll_version table is “development”.

So "development" builds of pgroll are isolated from versioned builds in terms of their effects on pgroll ’s internal state.

For semver comparisons.
Add a new table to track the pgroll binary version that last ran `pgroll
init`.
* On state initializtion, insert the current version of `pgroll` into
  the `pgroll_version` table.
* When creating a new `State` instance, check if the `pgroll` version
  matches the version stored in the `pgroll_version` table. If there is
  a version mismatch, re-run `Init`.
Wherever `State` instances are created we now pass the `pgroll` version.
Add testcases to cover `pgroll` schema upgrade scenarios when creating
`State` instances with different pgroll versions.
@github-actions github-actions bot temporarily deployed to Docs Preview June 10, 2025 15:38 Inactive
Copy link

Merging this branch will increase overall coverage

Impacted Packages Coverage Δ 🤖
github.com/xataio/pgroll/cmd 0.00% (ø)
github.com/xataio/pgroll/internal/testutils 0.00% (ø)
github.com/xataio/pgroll/pkg/state 55.69% (+9.05%) 👍

Coverage by file

Changed files (no unit tests)

Changed File Coverage Δ Total Covered Missed 🤖
github.com/xataio/pgroll/cmd/analyze.go 0.00% (ø) 15 0 15
github.com/xataio/pgroll/cmd/root.go 0.00% (ø) 55 0 55
github.com/xataio/pgroll/cmd/status.go 0.00% (ø) 15 0 15
github.com/xataio/pgroll/internal/testutils/util.go 0.00% (ø) 145 (+9) 0 145 (+9)
github.com/xataio/pgroll/pkg/state/opts.go 100.00% (+100.00%) 2 (+2) 2 (+2) 0 🌟
github.com/xataio/pgroll/pkg/state/state.go 44.38% (+3.39%) 178 (+17) 79 (+13) 99 (+4) 👍
github.com/xataio/pgroll/pkg/state/version.go 86.05% (+86.05%) 43 (+43) 37 (+37) 6 (+6) 🌟

Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code.

Changed unit test files

  • github.com/xataio/pgroll/pkg/state/state_test.go

@andrew-farries andrew-farries marked this pull request as ready for review June 10, 2025 15:49
@andrew-farries andrew-farries requested a review from kvch June 10, 2025 15:50
Copy link
Contributor

@kvch kvch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers, can go in as is.

@andrew-farries andrew-farries merged commit 192f4eb into main Jun 12, 2025
30 checks passed
@andrew-farries andrew-farries deleted the auto-init-old-pgroll-schema branch June 12, 2025 07:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants