{"record":{"id":"726fb7673a24c7e7","repo":"juanfont/headscale","slug":"setting-auth-key-to-null-on-nodes-with-non-existin","errorCode":null,"errorMessage":"setting auth_key to null on nodes with non-existing keys: %w","messagePattern":"setting auth_key to null on nodes with non-existing keys: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"hscontrol/db/db.go","lineNumber":137,"sourceCode":"\t\t\t\t\treturn nil\n\t\t\t\t},\n\t\t\t\tRollback: func(db *gorm.DB) error { return nil },\n\t\t\t},\n\t\t\t// Ensure there are no nodes referring to a deleted preauthkey.\n\t\t\t{\n\t\t\t\tID: \"202502070949\",\n\t\t\t\tMigrate: func(tx *gorm.DB) error {\n\t\t\t\t\tif tx.Migrator().HasTable(&types.PreAuthKey{}) {\n\t\t\t\t\t\terr := tx.Exec(`\nUPDATE nodes\nSET auth_key_id = NULL\nWHERE auth_key_id IS NOT NULL\nAND auth_key_id NOT IN (\n    SELECT id FROM pre_auth_keys\n);\n\t\t\t\t\t\t\t`).Error\n\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\treturn fmt.Errorf(\"setting auth_key to null on nodes with non-existing keys: %w\", err)\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\n\t\t\t\t\treturn nil\n\t\t\t\t},\n\t\t\t\tRollback: func(db *gorm.DB) error { return nil },\n\t\t\t},\n\t\t\t// v0.26.0\n\t\t\t// Migrate all routes from the Route table to the new field ApprovedRoutes\n\t\t\t// in the Node table. Then drop the Route table.\n\t\t\t{\n\t\t\t\tID: \"202502131714\",\n\t\t\t\tMigrate: func(tx *gorm.DB) error {\n\t\t\t\t\tif !tx.Migrator().HasColumn(&types.Node{}, \"approved_routes\") {\n\t\t\t\t\t\terr := tx.Migrator().AddColumn(&types.Node{}, \"approved_routes\")\n\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\treturn fmt.Errorf(\"adding column types.Node: %w\", err)\n\t\t\t\t\t\t}","sourceCodeStart":119,"sourceCodeEnd":155,"githubUrl":"https://github.com/juanfont/headscale/blob/565fd254d06c4c7f9a8cad1714a43445c79ba420/hscontrol/db/db.go#L119-L155","documentation":"Migration 202502070949 failed while nulling auth_key_id on nodes that reference non-existent pre-auth keys. The raw UPDATE ... WHERE auth_key_id NOT IN (SELECT id FROM pre_auth_keys) only fails on infrastructure problems - the code already guards with HasTable, so the cause is a locked, read-only, or permission-restricted database.","triggerScenarios":"The UPDATE cannot acquire a write lock (SQLite 'database is locked', Postgres row locks held by another transaction), the database user lacks UPDATE on nodes, or the SQLite file/directory is read-only.","commonSituations":"Concurrent headscale CLI command running during server startup migration; database on a read-only mount; NFS locking issues with SQLite.","solutions":["Ensure only one headscale process runs during startup; stop CLI usage until migrations finish.","Check write permissions on the SQLite file AND its directory (WAL/journal files).","For Postgres, grant UPDATE on the nodes table to the migration role.","Retry startup after clearing the lock - the migration is idempotent within its transaction."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"if _, err := db.NewHeadscaleDatabase(cfg); err != nil {\n    if strings.Contains(err.Error(), \"setting auth_key to null\") {\n        // write lock contention: stop concurrent writers and restart once\n    }\n}","preventionTips":["Do not run 'headscale' CLI commands while the server is starting up and migrating.","Verify the SQLite directory is writable (journal files must be creatable).","Use systemd ordering so only one headscale instance runs at a time."],"tags":["database","migration","sql","locking","sqlite"],"backgroundTag":null,"analyzedSha":"565fd254d06c4c7f9a8cad1714a43445c79ba420","analyzedAt":"2026-08-15T13:12:30.133Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}