{"record":{"id":"5e865cfab415bc13","repo":"SigNoz/signoz","slug":"codetimeout","errorCode":"CodeTimeout","errorMessage":"timed out waiting for lock","messagePattern":"timed out waiting for lock","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/sqlmigrator/migrator.go","lineNumber":128,"sourceCode":"\treturn nil\n}\n\nfunc (migrator *migrator) Lock(ctx context.Context) error {\n\tif err := migrator.migrator.Lock(ctx); err == nil {\n\t\tmigrator.settings.Logger().InfoContext(ctx, \"acquired migration lock\", slog.String(\"dialect\", migrator.dialect))\n\t\treturn nil\n\t}\n\n\ttimer := time.NewTimer(migrator.config.Lock.Timeout)\n\tdefer timer.Stop()\n\n\tticker := time.NewTicker(migrator.config.Lock.Interval)\n\tdefer ticker.Stop()\n\n\tfor {\n\t\tselect {\n\t\tcase <-timer.C:\n\t\t\terr := errors.New(errors.TypeTimeout, errors.CodeTimeout, \"timed out waiting for lock\")\n\t\t\tmigrator.settings.Logger().ErrorContext(ctx, \"cannot acquire lock\", errors.Attr(err), slog.String(\"lock_timeout\", migrator.config.Lock.Timeout.String()), slog.String(\"dialect\", migrator.dialect))\n\t\t\treturn err\n\t\tcase <-ticker.C:\n\t\t\tvar err error\n\t\t\tif err = migrator.migrator.Lock(ctx); err == nil {\n\t\t\t\tmigrator.settings.Logger().InfoContext(ctx, \"acquired migration lock\", slog.String(\"dialect\", migrator.dialect))\n\t\t\t\treturn nil\n\t\t\t}\n\t\t\tmigrator.settings.Logger().ErrorContext(ctx, \"attempt to acquire lock failed\", errors.Attr(err), slog.String(\"lock_interval\", migrator.config.Lock.Interval.String()), slog.String(\"dialect\", migrator.dialect))\n\t\tcase <-ctx.Done():\n\t\t\treturn ctx.Err()\n\t\t}\n\t}\n}\n","sourceCodeStart":110,"sourceCodeEnd":143,"githubUrl":"https://github.com/SigNoz/signoz/blob/5069bf80b08f1f00d7e014eccc09902f9871004f/pkg/sqlmigrator/migrator.go#L110-L143","documentation":"Before running or rolling back migrations, the migrator polls an advisory lock every Lock.Interval until Lock.Timeout elapses. If the lock is still held (or the lock call keeps failing), the operation aborts with this timeout error and a log entry 'cannot acquire lock'.","triggerScenarios":"Concurrent deployments/migration jobs where one process holds the migration lock longer than the configured timeout, or a stale lock left by a crashed process, or the database lock call erroring (permissions, connectivity) each tick.","commonSituations":"Two CI/CD pipelines deploying simultaneously, a long-running migration blocking others, a previous migration process killed without releasing the lock, or Lock.Timeout configured too low.","solutions":["Increase Lock.Timeout so it exceeds the expected longest migration","Ensure only one migration job runs at a time (deploy serialization / leader election)","If a stale lock is suspected, clear the advisory lock (e.g. pg_advisory_unlock or terminate the holding session) after confirming no active migration","Check the error log for the underlying lock-call failures (dialect, permissions)"],"exampleFix":"// before\ncfg.Lock.Timeout = 10 * time.Second\n\n// after\ncfg.Lock.Timeout = 10 * time.Minute","handlingStrategy":"retry","validationCode":"// pre-flight: check no other migration holds the lock\n// and ensure timeout comfortably exceeds longest migration\nif cfg.Lock.Timeout <= longestExpectedMigration {\n    cfg.Lock.Timeout = 10 * time.Minute\n}","typeGuard":null,"tryCatchPattern":"err := m.Migrate(ctx)\nif err != nil && strings.Contains(err.Error(), \"timed out waiting for lock\") {\n    // backoff, verify no active migration, then retry once\n}","preventionTips":["Serialize migration jobs across environments (single deployer)","Set Lock.Timeout based on worst-case migration duration","Monitor for stale locks after crashed migrator processes"],"tags":["go","sql-migrator","advisory-lock","timeout","concurrency"],"backgroundTag":"lock-acquisition-timeout","analyzedSha":"5069bf80b08f1f00d7e014eccc09902f9871004f","analyzedAt":"2026-08-28T06:22:12.824Z","schemaVersion":2},"datasetVersion":"2026-08-28T11:17:15.048Z"}