Skip to content

Add support for exclusion constraints in create_table operation #624

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 71 commits into from
Jan 28, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
5908c19
mi folyik itt gyongyoson?
kvch Jan 9, 2025
c9ef10d
update generated file
kvch Jan 9, 2025
03b3952
first batch
kvch Jan 9, 2025
f84b9a7
more updates
kvch Jan 10, 2025
296731e
validation
kvch Jan 10, 2025
766ddbc
add table constraint
kvch Jan 10, 2025
965f28f
minor fixes
kvch Jan 10, 2025
abd6aed
add missing fixes
kvch Jan 10, 2025
b1dba4a
moreeee
kvch Jan 10, 2025
a0e918c
add default values
kvch Jan 10, 2025
1ef47fd
uppercase
kvch Jan 10, 2025
253e436
add validation test
kvch Jan 10, 2025
fd54808
rm messages
kvch Jan 13, 2025
b088ffd
deparse storage params
kvch Jan 13, 2025
e0acc68
init check constraints
kvch Jan 13, 2025
90c548d
Merge remote-tracking branch 'upstream/main' into feature-add-table-c…
kvch Jan 13, 2025
dab75c2
add deparsing
kvch Jan 13, 2025
cb7b922
add docs
kvch Jan 13, 2025
00bb93f
inimini
kvch Jan 13, 2025
cf25acc
even more
kvch Jan 13, 2025
f33c2ea
rm unnecessary test
kvch Jan 14, 2025
8f9ecf6
update pr
kvch Jan 14, 2025
f187b8b
fmt
kvch Jan 14, 2025
1e527e7
add missing updates
kvch Jan 14, 2025
f292e82
initialize primary key constraint
kvch Jan 14, 2025
f432d75
Merge remote-tracking branch 'upstream/main' into feature-add-table-c…
kvch Jan 16, 2025
d788e69
add jsonschema tests
kvch Jan 16, 2025
3e54252
rm duplicate
kvch Jan 16, 2025
da60072
edit comments
kvch Jan 16, 2025
981df44
add support for foreign key constraints
kvch Jan 16, 2025
40dc814
add tests
kvch Jan 16, 2025
cbf5043
add e2e
kvch Jan 16, 2025
b63d0e0
update test
kvch Jan 16, 2025
fcfdd15
Merge remote-tracking branch 'upstream/main' into feature-add-table-c…
kvch Jan 17, 2025
f0913a3
add example
kvch Jan 17, 2025
c82f830
minor fix
kvch Jan 17, 2025
385e7d5
add jsonschema tests
kvch Jan 17, 2025
37393e9
update test
kvch Jan 17, 2025
be2830c
fix table name
kvch Jan 17, 2025
1e0d403
pass match_type to schema.foreignkey
kvch Jan 20, 2025
c30e0c5
Merge remote-tracking branch 'upstream/main' into feature-add-table-c…
kvch Jan 21, 2025
d6f9cf7
Address review notes and add new attribute
kvch Jan 21, 2025
1a9eb26
Document new option on_delete_set_columns
kvch Jan 21, 2025
0ee70ea
Add more assertions to e2e test
kvch Jan 21, 2025
b47ade4
fix
kvch Jan 21, 2025
eb3965d
adjust expected
kvch Jan 21, 2025
115af43
mr
kvch Jan 21, 2025
ed04cdf
improve validation
kvch Jan 21, 2025
1da339a
add missing file
kvch Jan 21, 2025
2a76649
support setting columns
kvch Jan 22, 2025
6e03cb5
skip new tests
kvch Jan 22, 2025
b30a1b5
mivan?
kvch Jan 22, 2025
3b0d39e
put support in the correct place
kvch Jan 22, 2025
312cf21
init
kvch Jan 22, 2025
ea0e487
Merge remote-tracking branch 'upstream/main' into feature-add-table-c…
kvch Jan 23, 2025
51ce6a5
add exclude constraint
kvch Jan 24, 2025
6c448e7
even more
kvch Jan 24, 2025
8e060ce
fix test
kvch Jan 24, 2025
2c387fb
add missing extension
kvch Jan 24, 2025
a141333
always extension
kvch Jan 24, 2025
c1e11ce
install extension
kvch Jan 24, 2025
e57c361
most jo lesz?
kvch Jan 24, 2025
5bf85b6
miert
kvch Jan 24, 2025
e30a35e
clean
kvch Jan 24, 2025
7004ff5
move
kvch Jan 24, 2025
d9e51b5
update dep
kvch Jan 24, 2025
76122d2
more
kvch Jan 24, 2025
3a90684
rm extension
kvch Jan 27, 2025
1d301bb
fix name
kvch Jan 27, 2025
b943ed5
rename example table
kvch Jan 27, 2025
9e5d6a4
really fix example
kvch Jan 27, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion docs/operations/create_table.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -76,10 +76,15 @@ Each `constraint` is defined as:
"storage_parameters": "parameter=value",
"include_columns": ["list", "of", "columns", "included in index"]
},
"exclude": {
"index_method": "name of the index method, e.g. btree",
"elements": "exclude elements",
"predicate": "WHERE clause of the exclude constraint"
}
},
```

Supported constraint types: `unique`, `check`, `primary_key`, `foreign_key`.
Supported constraint types: `unique`, `check`, `primary_key`, `foreign_key`, `exclude`.

Please note that you can only configure primary keys in `columns` list or `constraints` list, but
not in both places.
Expand Down Expand Up @@ -154,3 +159,9 @@ Create a table with table level constraints:
Create a table with table level foreign key constraints:

<ExampleSnippet example="51_create_table_with_table_foreign_key_constraint.json" language="json" />

### Create a table with exclusion constraint

Create a table with an exclusion:

<ExampleSnippet example="52_create_table_with_table_exclusion_constraint.json" language="json" />
1 change: 1 addition & 0 deletions examples/.ledger
Original file line number Diff line number Diff line change
Expand Up @@ -49,3 +49,4 @@
49_unset_not_null_on_indexed_column.json
50_create_table_with_table_constraint.json
51_create_table_with_table_foreign_key_constraint.json
52_create_table_with_exclusion_constraint.json
46 changes: 46 additions & 0 deletions examples/52_create_table_with_exclusion_constraint.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
{
"name": "52_create_table_with_table_exclusion_constraint",
"operations": [
{
"create_table": {
"name": "library",
"columns": [
{
"name": "id",
"type": "serial"
},
{
"name": "returned",
"type": "timestamp"
},
{
"name": "title",
"type": "text"
},
{
"name": "summary",
"type": "text"
}
],
"constraints": [
{
"name": "rooms_pk",
"type": "primary_key",
"columns": [
"id"
]
},
{
"name": "forbid_duplicated_titles",
"type": "exclude",
"exclude": {
"index_method": "btree",
"elements": "title WITH =",
"predicate": "title IS NOT NULL"
}
}
]
}
}
]
}
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ require (
github.com/stretchr/testify v1.10.0
github.com/testcontainers/testcontainers-go v0.35.0
github.com/testcontainers/testcontainers-go/modules/postgres v0.35.0
github.com/xataio/pg_query_go/v6 v6.0.0-20250122133641-54118c062181
github.com/xataio/pg_query_go/v6 v6.0.0-20250124115938-4fa82ad6d036
golang.org/x/tools v0.29.0
)

Expand Down
4 changes: 4 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,10 @@ github.com/tklauser/numcpus v0.8.0 h1:Mx4Wwe/FjZLeQsK/6kt2EOepwwSl7SmJrK5bV/dXYg
github.com/tklauser/numcpus v0.8.0/go.mod h1:ZJZlAY+dmR4eut8epnzf0u/VwodKmryxR8txiloSqBE=
github.com/xataio/pg_query_go/v6 v6.0.0-20250122133641-54118c062181 h1:iLOHgul20WFUhO4eFpJ/lZRkHzZICF2ghzncxtOcD0E=
github.com/xataio/pg_query_go/v6 v6.0.0-20250122133641-54118c062181/go.mod h1:GK6bpfAhPtZb7wG/IccqvnH+cz3cmvvRTkC+MosESGo=
github.com/xataio/pg_query_go/v6 v6.0.0-20250123182324-526a22cbe0c0 h1:/fXLU7NusFv8TSEAM7n+vtsN5Rr2La9tXWEunrJOWrI=
github.com/xataio/pg_query_go/v6 v6.0.0-20250123182324-526a22cbe0c0/go.mod h1:GK6bpfAhPtZb7wG/IccqvnH+cz3cmvvRTkC+MosESGo=
github.com/xataio/pg_query_go/v6 v6.0.0-20250124115938-4fa82ad6d036 h1:QMjBW1XIFUDzyUs/zlYmUo8lNO+hQ4VVt0msFv2AYpw=
github.com/xataio/pg_query_go/v6 v6.0.0-20250124115938-4fa82ad6d036/go.mod h1:GK6bpfAhPtZb7wG/IccqvnH+cz3cmvvRTkC+MosESGo=
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778/go.mod h1:2MuV+tbUrU1zIOPMxZ5EncGwgmMJsa+9ucAQZXxsObs=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM=
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
This is an invalid 'create_table' migration.
Exclusion constraints must have exclude configured

-- create_table.json --
{
"name": "migration_name",
"operations": [
{
"create_table": {
"name": "posts",
"columns": [
{
"name": "title",
"type": "varchar(255)"
},
{
"name": "user_id",
"type": "integer",
"nullable": true
}
],
"constraints": [
{
"name": "my_invalid_fk",
"type": "exclude"
}
]
}
}
]
}

-- valid --
false
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
This is an invalid 'create_table' migration.
Exclusion constraints mustn't have columns configured

-- create_table.json --
{
"name": "migration_name",
"operations": [
{
"create_table": {
"name": "posts",
"columns": [
{
"name": "title",
"type": "varchar(255)"
},
{
"name": "user_id",
"type": "integer",
"nullable": true
}
],
"constraints": [
{
"name": "my_invalid_exclusion",
"type": "exclude",
"columns": ["invalid"],
"exclude": {
"index_method": "btree",
"elements": "title WITH ="
}
}
]
}
}
]
}

-- valid --
false
13 changes: 7 additions & 6 deletions internal/testutils/error_codes.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,11 @@
package testutils

const (
CheckViolationErrorCode string = "check_violation"
FKViolationErrorCode string = "foreign_key_violation"
NotNullViolationErrorCode string = "not_null_violation"
UniqueViolationErrorCode string = "unique_violation"
UndefinedColumnErrorCode string = "undefined_column"
UndefinedTableErrorCode string = "undefined_table"
CheckViolationErrorCode string = "check_violation"
ExclusionViolationErrorCode string = "exclusion_violation"
FKViolationErrorCode string = "foreign_key_violation"
NotNullViolationErrorCode string = "not_null_violation"
UndefinedColumnErrorCode string = "undefined_column"
UndefinedTableErrorCode string = "undefined_table"
UniqueViolationErrorCode string = "unique_violation"
)
27 changes: 27 additions & 0 deletions pkg/migrations/op_common_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -272,6 +272,13 @@ func PrimaryKeyConstraintMustExist(t *testing.T, db *sql.DB, schema, table, cons
}
}

func ExcludeConstraintMustExist(t *testing.T, db *sql.DB, schema, table, constraint string) {
t.Helper()
if !excludeConstraintExists(t, db, schema, table, constraint) {
t.Fatalf("Expected constraint %q to exist", constraint)
}
}

func IndexMustExist(t *testing.T, db *sql.DB, schema, table, index string) {
t.Helper()
if !indexExists(t, db, schema, table, index) {
Expand Down Expand Up @@ -477,6 +484,26 @@ func primaryKeyConstraintExists(t *testing.T, db *sql.DB, schema, table, constra
return exists
}

func excludeConstraintExists(t *testing.T, db *sql.DB, schema, table, constraint string) bool {
t.Helper()

var exists bool
err := db.QueryRow(`
SELECT EXISTS (
SELECT 1
FROM pg_catalog.pg_constraint
WHERE conrelid = $1::regclass
AND conname = $2
AND contype = 'x'
)`,
fmt.Sprintf("%s.%s", schema, table), constraint).Scan(&exists)
if err != nil {
t.Fatal(err)
}

return exists
}

func triggerExists(t *testing.T, db *sql.DB, schema, table, trigger string) bool {
t.Helper()

Expand Down
37 changes: 31 additions & 6 deletions pkg/migrations/op_create_table.go
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,7 @@ func (o *OpCreateTable) updateSchema(s *schema.Schema) *schema.Schema {
uniqueConstraints := make(map[string]*schema.UniqueConstraint, 0)
checkConstraints := make(map[string]*schema.CheckConstraint, 0)
foreignKeys := make(map[string]*schema.ForeignKey, 0)
excludeConstraints := make(map[string]*schema.ExcludeConstraint, 0)
for _, c := range o.Constraints {
switch c.Type {
case ConstraintTypeUnique:
Expand All @@ -240,16 +241,24 @@ func (o *OpCreateTable) updateSchema(s *schema.Schema) *schema.Schema {
OnUpdate: string(c.References.OnUpdate),
MatchType: string(c.References.MatchType),
}
case ConstraintTypeExclude:
excludeConstraints[c.Name] = &schema.ExcludeConstraint{
Name: c.Name,
IndexMethod: c.Exclude.IndexMethod,
Elements: c.Exclude.Elements,
Predicate: c.Exclude.Predicate,
}
}
}

s.AddTable(o.Name, &schema.Table{
Name: o.Name,
Columns: columns,
UniqueConstraints: uniqueConstraints,
CheckConstraints: checkConstraints,
PrimaryKey: primaryKeys,
ForeignKeys: foreignKeys,
Name: o.Name,
Columns: columns,
UniqueConstraints: uniqueConstraints,
CheckConstraints: checkConstraints,
PrimaryKey: primaryKeys,
ForeignKeys: foreignKeys,
ExcludeConstraints: excludeConstraints,
})

return s
Expand Down Expand Up @@ -311,6 +320,8 @@ func constraintsToSQL(constraints []Constraint) (string, error) {
constraintsSQL[i] = writer.WritePrimaryKey()
case ConstraintTypeForeignKey:
constraintsSQL[i] = writer.WriteForeignKey(c.References.Table, c.References.Columns, c.References.OnDelete, c.References.OnUpdate, c.References.OnDeleteSetColumns)
case ConstraintTypeExclude:
constraintsSQL[i] = writer.WriteExclude(c.Exclude.IndexMethod, c.Exclude.Elements, c.Exclude.Predicate)
}
}
if len(constraintsSQL) == 0 {
Expand Down Expand Up @@ -397,6 +408,20 @@ func (w *ConstraintSQLWriter) WriteForeignKey(referencedTable string, referenced
return constraint
}

func (w *ConstraintSQLWriter) WriteExclude(indexMethod, elements, predicate string) string {
constraint := ""
if w.Name != "" {
constraint = fmt.Sprintf("CONSTRAINT %s ", pq.QuoteIdentifier(w.Name))
}
constraint += fmt.Sprintf("EXCLUDE USING %s (%s)", indexMethod, elements)
constraint += w.addIndexParameters()
if predicate != "" {
constraint += fmt.Sprintf(" WHERE (%s)", predicate)
}
constraint += w.addDeferrable()
return constraint
}

func (w *ConstraintSQLWriter) addIndexParameters() string {
constraint := ""
if len(w.IncludeColumns) != 0 {
Expand Down
70 changes: 70 additions & 0 deletions pkg/migrations/op_create_table_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1020,6 +1020,76 @@ func TestCreateTable(t *testing.T) {
}, rows)
},
},
{
name: "create table with a simple exclude constraint mimicking unique constraint",
migrations: []migrations.Migration{
{
Name: "01_create_table",
Operations: migrations.Operations{
&migrations.OpCreateTable{
Name: "users",
Columns: []migrations.Column{
{
Name: "id",
Type: "int",
},
{
Name: "name",
Type: "text",
},
},
Constraints: []migrations.Constraint{
{
Name: "exclude_id",
Type: migrations.ConstraintTypeExclude,
Exclude: &migrations.ConstraintExclude{
IndexMethod: "btree",
Elements: "id WITH =",
},
},
},
},
},
},
},
afterStart: func(t *testing.T, db *sql.DB, schema string) {
// The exclusion constraint exists on the new table.
ExcludeConstraintMustExist(t, db, schema, "users", "exclude_id")

// Inserting a row into the table succeeds when the exclude constraint is satisfied.
// This is the first row, so the constraint is satisfied.
MustInsert(t, db, schema, "01_create_table", "users", map[string]string{
"id": "1",
"name": "alice",
})

// Inserting a row into the table fails because there is already a row with id 1.
MustNotInsert(t, db, schema, "01_create_table", "users", map[string]string{
"id": "1",
"name": "b",
}, testutils.ExclusionViolationErrorCode)
},
afterRollback: func(t *testing.T, db *sql.DB, schema string) {
// The table has been dropped, so the constraint is gone.
},
afterComplete: func(t *testing.T, db *sql.DB, schema string) {
// The exclusion constraint exists on the new table.
ExcludeConstraintMustExist(t, db, schema, "users", "exclude_id")

// Inserting a row into the table succeeds when the exclude constraint is satisfied.
// This is the first row, so the constraint is satisfied.
MustInsert(t, db, schema, "01_create_table", "users", map[string]string{
"id": "1",
"name": "alice",
})

// Inserting a row into the table fails because there is already a row with id 1.
MustNotInsert(t, db, schema, "01_create_table", "users", map[string]string{
"id": "1",
"name": "b",
}, testutils.ExclusionViolationErrorCode)
},
},
})
}

Expand Down
Loading