{"record":{"id":"005ee72e11d7f503","repo":"hashicorp/nomad","slug":"acl-policy-lookup-failed-v-005ee7","errorCode":null,"errorMessage":"ACL policy lookup failed: %v","messagePattern":"ACL policy lookup failed: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/state/state_store_acl.go","lineNumber":169,"sourceCode":"\t} else {\n\t\trole.CreateIndex = index\n\t\trole.ModifyIndex = index\n\t}\n\n\t// Insert the role into the table.\n\tif err := txn.Insert(TableACLRoles, role); err != nil {\n\t\treturn false, fmt.Errorf(\"ACL role insert failed: %v\", err)\n\t}\n\treturn true, nil\n}\n\n// validateACLRolePolicyLinksTxn is the same as ValidateACLRolePolicyLinks but\n// allows callers to pass their own transaction.\nfunc (s *StateStore) validateACLRolePolicyLinksTxn(txn *txn, role *structs.ACLRole) error {\n\tfor _, policyLink := range role.Policies {\n\t\t_, existing, err := txn.FirstWatch(\"acl_policy\", indexID, policyLink.Name)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"ACL policy lookup failed: %v\", err)\n\t\t}\n\t\tif existing == nil {\n\t\t\treturn errors.New(\"ACL policy not found\")\n\t\t}\n\t}\n\treturn nil\n}\n\n// DeleteACLRolesByID is responsible for batch deleting ACL roles based on\n// their ID. It uses a single write transaction for efficiency, however, any\n// error means no entries will be committed. An error is produced if a role is\n// not found within state which has been passed within the array.\nfunc (s *StateStore) DeleteACLRolesByID(\n\tmsgType structs.MessageType, index uint64, roleIDs []string) error {\n\n\ttxn := s.db.WriteTxnMsgT(msgType, index)\n\tdefer txn.Abort()\n","sourceCodeStart":151,"sourceCodeEnd":187,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/state/state_store_acl.go#L151-L187","documentation":"During ACL role validation, each policy link in role.Policies is looked up by name in the acl_policy table; this error wraps a memdb read failure (not a missing policy — a missing policy yields 'ACL policy not found'). It indicates the policy-existence check itself failed at the transaction level, aborting the role upsert.","triggerScenarios":"upsertACLRoleTxn (via UpsertACLRoles) when txn.FirstWatch(\"acl_policy\", \"id\", policyLink.Name) returns a non-nil error — corrupted txn/table state, or patched code using a wrong index name.","commonSituations":"Corrupted server state store; embedded/modified Nomad where acl_policy index names changed; race with table rebuild in unusual deployments.","solutions":["Retry the upsert after confirming the server is healthy and is the leader.","Restart the Nomad server agent to rebuild the in-memory store from raft.","Check server logs for prior memdb errors indicating corruption; restore from backup if needed.","If running patched code, verify acl_policy table uses indexID (\"id\") for name lookups.","Distinguish from the sibling 'ACL policy not found' error: this one is a system fault, not a config mistake."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"policies, _, _ := client.ACL().Policies().List(nil)\nexisting := map[string]bool{}\nfor _, p := range policies { existing[p.Name] = true }\nfor _, link := range role.Policies {\n    if !existing[link.Name] { return fmt.Errorf(\"policy %q does not exist\", link.Name) }\n}","typeGuard":null,"tryCatchPattern":"_, _, err := client.ACL().Roles().Create(role, nil)\nif err != nil {\n    if strings.Contains(err.Error(), \"ACL policy not found\") { /* config fix: create the policy */ }\n    if strings.Contains(err.Error(), \"ACL policy lookup failed\") { /* system fault: retry/restart */ }\n}","preventionTips":["Validate every policy link exists before upserting a role.","Create referenced policies and roles in dependency order in IaC.","Distinguish 'lookup failed' (system fault) from 'policy not found' (config error).","Retry after server health checks rather than altering config on this error."],"tags":["nomad","acl","state-store","memdb"],"backgroundTag":"state-store-transaction-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}