What causes "unable to obtain lock" in golang-migrate/migrate?

Ensure only one migration runner executes at a time (serialize deploys, use a job queue or leader election).

unable to obtain lock

Error message

unable to obtain lock

What it means

spanner.ErrLockHeld ('unable to obtain lock') is returned by the Spanner driver's Lock method when another process already holds the advisory migration lock for the database. The driver uses the lock so only one migrate instance mutates the schema at a time; failing to acquire it returns this sentinel.

Source

Thrown at database/spanner/spanner.go:40

	adminpb "cloud.google.com/go/spanner/admin/database/apiv1/databasepb"
	"google.golang.org/api/iterator"
)

func init() {
	db := Spanner{}
	database.Register("spanner", &db)
}

// DefaultMigrationsTable is used if no custom table is specified
const DefaultMigrationsTable = "SchemaMigrations"

// Driver errors
var (
	ErrNilConfig      = errors.New("no config")
	ErrNoDatabaseName = errors.New("no database name")
	ErrNoSchema       = errors.New("no schema")
	ErrDatabaseDirty  = errors.New("database is dirty")
	ErrLockHeld       = errors.New("unable to obtain lock")
	ErrLockNotHeld    = errors.New("unable to release already released lock")
)

// Config used for a Spanner instance
type Config struct {
	MigrationsTable string
	DatabaseName    string
	// Whether to parse the migration DDL with spansql before
	// running them towards Spanner.
	// Parsing outputs clean DDL statements such as reformatted
	// and void of comments.
	CleanStatements bool
}

// Spanner implements database.Driver for Google Cloud Spanner
type Spanner struct {
	db *DB

View on GitHub (pinned to 01a9643f14)

Solutions

Ensure only one migration runner executes at a time (serialize deploys, use a job queue or leader election).
Wait and retry Lock until the current holder finishes (migrate's Lock retries internally; add backoff around your run).
If the holder is definitely dead, manually clear the lock state the driver persists, then retry.

Example fix

// before
driver, _ := spanner.WithInstance(client, cfg) // Lock() fails if another runner holds it
// after
if err := database.SetMigrationLockTimeout(...); err != nil { ... }
// or run migrations exclusively:
if err := driver.Lock(); err != nil {
    if errors.Is(err, spanner.ErrLockHeld) {
        time.Sleep(10 * time.Second)
        err = driver.Lock()
    }
}

Defensive patterns

Strategy: retry

Validate before calling

if err := driver.Lock(); err != nil {
    if errors.Is(err, spanner.ErrLockHeld) {
        return fmt.Errorf("another migration is running; retry later")
    }
    return err
}

Try / catch

err := retry.Do(func() error {
    if lerr := driver.Lock(); lerr != nil {
        if errors.Is(lerr, spanner.ErrLockHeld) {
            return retry.RetryableError(lerr)
        }
        return retry.UnrecoverableError(lerr)
    }
    return nil
}, retry.Attempts(5), retry.Delay(10*time.Second))

Prevention

Serialize deploys so only one migration runner targets a database at a time.
Set a bounded lock timeout/backoff instead of failing the deploy instantly.
Alert on ErrLockHeld to detect concurrent pipelines or stale locks.
Clean up lock state when a migration runner is definitively dead.

When it happens

Trigger: Two migrate processes (or two deploy jobs) running concurrently against the same Spanner database; a previous holder crashed without unlocking and its lock row/lease is still considered active.

Common situations: Parallel CI pipelines deploying the same service, blue/green deploys where both sides run migrations on startup, a stale lock left by a killed migration container.

Related errors

AI-assisted analysis of golang-migrate/migrate@01a9643f14 (2026-09-02). Data as JSON: /api/errors/0371103f176df72b. Report an issue: GitHub.