{"record":{"id":"deb391c641e7f89f","repo":"hashicorp/nomad","slug":"deleting-acl-token-failed-v","errorCode":null,"errorMessage":"deleting acl token failed: %v","messagePattern":"deleting acl token failed: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/state/state_store.go","lineNumber":6424,"sourceCode":"\t\t}\n\t}\n\n\t// Update the indexes table\n\tif err := txn.Insert(\"index\", &IndexEntry{\"acl_token\", index}); err != nil {\n\t\treturn fmt.Errorf(\"index update failed: %v\", err)\n\t}\n\treturn txn.Commit()\n}\n\n// DeleteACLTokens deletes the tokens with the given accessor ids\nfunc (s *StateStore) DeleteACLTokens(msgType structs.MessageType, index uint64, ids []string) error {\n\ttxn := s.db.WriteTxnMsgT(msgType, index)\n\tdefer txn.Abort()\n\n\t// Delete the tokens\n\tfor _, id := range ids {\n\t\tif _, err := txn.DeleteAll(\"acl_token\", \"id\", id); err != nil {\n\t\t\treturn fmt.Errorf(\"deleting acl token failed: %v\", err)\n\t\t}\n\t}\n\tif err := txn.Insert(\"index\", &IndexEntry{\"acl_token\", index}); err != nil {\n\t\treturn fmt.Errorf(\"index update failed: %v\", err)\n\t}\n\treturn txn.Commit()\n}\n\n// ACLTokenByAccessorID is used to lookup a token by accessor ID\nfunc (s *StateStore) ACLTokenByAccessorID(ws memdb.WatchSet, id string) (*structs.ACLToken, error) {\n\tif id == \"\" {\n\t\treturn nil, fmt.Errorf(\"acl token lookup failed: missing accessor id\")\n\t}\n\n\ttxn := s.db.ReadTxn()\n\n\twatchCh, existing, err := txn.FirstWatch(\"acl_token\", \"id\", id)\n\tif err != nil {","sourceCodeStart":6406,"sourceCodeEnd":6442,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/state/state_store.go#L6406-L6442","documentation":"Thrown in StateStore.DeleteACLTokens when txn.DeleteAll on the 'acl_token' table by accessor ID fails. The delete transaction aborts, so no tokens are removed and the Raft log entry fails, surfacing as a failed ACLTokenDelete RPC.","triggerScenarios":"DeleteACLTokens with a memdb error on 'acl_token'/'id' — schema drift for the table, store closed concurrently, or internal corruption of the in-memory table.","commonSituations":"Deleting expired/revoked tokens during a version upgrade with schema skew; forked builds with altered acl_token indexes; tests using a torn-down state store.","solutions":["Restart the Nomad server to rebuild the state store from BoltDB.","Confirm the acl_token table defines the 'id' index in the deployed schema version.","Check the wrapped %v cause in logs; restore server state from backup if corruption is indicated."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"func validDeleteIDs(ids []string) bool {\n    for _, id := range ids {\n        if id == \"\" { return false }\n    }\n    return len(ids) > 0\n}","typeGuard":null,"tryCatchPattern":"if err := store.DeleteACLTokens(msgType, index, ids); err != nil {\n    if strings.Contains(err.Error(), \"deleting acl token failed\") {\n        return fmt.Errorf(\"state store token delete aborted: %w\", err)\n    }\n    return err\n}","preventionTips":["Validate IDs are non-empty before the delete RPC","Restart the server on state store errors; do not retry the same Raft write blindly","Keep acl_token 'id' index definitions in sync with the binary","Track delete failures via ACLTokenDelete RPC error metrics"],"tags":["nomad","state-store","acl","raft"],"backgroundTag":"state-store-write-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}