{"record":{"id":"b79844c3bcd65747","repo":"juanfont/headscale","slug":"clearing-expiry-on-tagged-nodes-w","errorCode":null,"errorMessage":"clearing expiry on tagged nodes: %w","messagePattern":"clearing expiry on tagged nodes: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"hscontrol/db/db.go","lineNumber":923,"sourceCode":"\t\t\t\t// owned by its tags and never expires (KB 1068), but a buggy\n\t\t\t\t// handleLogout stamped a past expiry on it, leaving it\n\t\t\t\t// permanently Expired and unable to re-authenticate. The\n\t\t\t\t// buggy writer is fixed, so this only repairs rows written\n\t\t\t\t// before the upgrade; a fixed server cannot recreate them.\n\t\t\t\t// Match the tagged-node predicate the earlier\n\t\t\t\t// clear-tagged-node-user-id migration uses (a nil tags slice\n\t\t\t\t// marshals to 'null', so exclude it).\n\t\t\t\t// Fixes: https://github.com/juanfont/headscale/issues/3371\n\t\t\t\tID: \"202607241200-clear-tagged-node-expiry\",\n\t\t\t\tMigrate: func(tx *gorm.DB) error {\n\t\t\t\t\terr := tx.Exec(`\nUPDATE nodes\nSET expiry = NULL\nWHERE tags IS NOT NULL AND tags != '[]' AND tags != '' AND tags != 'null'\n\tAND expiry IS NOT NULL;\n\t\t\t\t\t\t`).Error\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\treturn fmt.Errorf(\"clearing expiry on tagged nodes: %w\", err)\n\t\t\t\t\t}\n\n\t\t\t\t\treturn nil\n\t\t\t\t},\n\t\t\t\tRollback: func(db *gorm.DB) error { return nil },\n\t\t\t},\n\t\t},\n\t)\n\n\tmigrations.InitSchema(func(tx *gorm.DB) error {\n\t\t// Create all tables using AutoMigrate\n\t\terr := tx.AutoMigrate(\n\t\t\t&types.User{},\n\t\t\t&types.PreAuthKey{},\n\t\t\t&types.APIKey{},\n\t\t\t&types.Node{},\n\t\t\t&types.Policy{},\n\t\t\t&types.OAuthClient{},","sourceCodeStart":905,"sourceCodeEnd":941,"githubUrl":"https://github.com/juanfont/headscale/blob/565fd254d06c4c7f9a8cad1714a43445c79ba420/hscontrol/db/db.go#L905-L941","documentation":"Thrown inside GORM migration '202607241200-clear-tagged-node-expiry', which nulls out the expiry column on tagged nodes (tagged nodes cannot expire, per headscale's tags-XOR-user-ownership model). The wrapping error means the raw UPDATE statement failed at the SQL level. The underlying driver error is chained via %w and tells the real story.","triggerScenarios":"Running headscale startup (or an explicit migrate) against a database where the UPDATE nodes SET expiry = NULL ... statement cannot execute: table 'nodes' missing/recreated, sqlite database file locked by another process, postgres connection dropped mid-migration, or the tags column has an unexpected type making the comparison invalid.","commonSituations":"Upgrading headscale to a version containing this migration while another headscale/litestream instance holds the sqlite file; a partially-restored or hand-edited database where the nodes table was dropped; running migrations on a read-only filesystem or full disk.","solutions":["Check the chained driver error in the same log line — it names the exact SQL failure (locked, no such table, disk I/O).","Ensure only one headscale process is running against the sqlite path; stop litestream or take a backup before migrating.","Verify filesystem permissions and free space on the volume holding cfg.Database.Sqlite.Path.","If the nodes table is missing/corrupt, restore from backup (or litestream replica) and re-run startup so the migration retries.","Confirm the database was not created by a much older/newer headscale with an incompatible nodes schema; migrate stepwise through intermediate versions."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Before startup: ensure the sqlite file is not held by another process\nimport \"os\"\n\nfunc canLock(path string) bool {\n    f, err := os.OpenFile(path+\".probe\", os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o644)\n    if err != nil { return false }\n    f.Close()\n    os.Remove(path + \".probe\")\n    return true\n}","typeGuard":null,"tryCatchPattern":"// headscale surfaces this at startup and exits; handle at the orchestrator level:\n// systemd unit with Restart=on-failure and a backoff gives lock-transient\n// migration failures a chance to succeed on retry. Inspect the wrapped %w\n// error in logs to distinguish 'database is locked' (retry helps) from\n// SQL/logic errors (retry will not help).","preventionTips":["Run exactly one headscale instance per database file.","Take a backup (or ensure litestream replication is current) before every headscale upgrade.","Never edit or reorder committed migrations; only append new ones.","Keep binary version and database schema in lockstep — avoid downgrades."],"tags":["database","migration","sqlite","postgres","startup"],"backgroundTag":null,"analyzedSha":"565fd254d06c4c7f9a8cad1714a43445c79ba420","analyzedAt":"2026-08-15T13:12:30.133Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}