hashicorp/nomad · error

ACL policy lookup failed: %v

Error message

ACL policy lookup failed: %v

What it means

During ACL role validation, each policy link in role.Policies is looked up by name in the acl_policy table; this error wraps a memdb read failure (not a missing policy — a missing policy yields 'ACL policy not found'). It indicates the policy-existence check itself failed at the transaction level, aborting the role upsert.

Source

Thrown at nomad/state/state_store_acl.go:169

	} else {
		role.CreateIndex = index
		role.ModifyIndex = index
	}

	// Insert the role into the table.
	if err := txn.Insert(TableACLRoles, role); err != nil {
		return false, fmt.Errorf("ACL role insert failed: %v", err)
	}
	return true, nil
}

// validateACLRolePolicyLinksTxn is the same as ValidateACLRolePolicyLinks but
// allows callers to pass their own transaction.
func (s *StateStore) validateACLRolePolicyLinksTxn(txn *txn, role *structs.ACLRole) error {
	for _, policyLink := range role.Policies {
		_, existing, err := txn.FirstWatch("acl_policy", indexID, policyLink.Name)
		if err != nil {
			return fmt.Errorf("ACL policy lookup failed: %v", err)
		}
		if existing == nil {
			return errors.New("ACL policy not found")
		}
	}
	return nil
}

// DeleteACLRolesByID is responsible for batch deleting ACL roles based on
// their ID. It uses a single write transaction for efficiency, however, any
// error means no entries will be committed. An error is produced if a role is
// not found within state which has been passed within the array.
func (s *StateStore) DeleteACLRolesByID(
	msgType structs.MessageType, index uint64, roleIDs []string) error {

	txn := s.db.WriteTxnMsgT(msgType, index)
	defer txn.Abort()

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Retry the upsert after confirming the server is healthy and is the leader.
2. Restart the Nomad server agent to rebuild the in-memory store from raft.
3. Check server logs for prior memdb errors indicating corruption; restore from backup if needed.
4. If running patched code, verify acl_policy table uses indexID ("id") for name lookups.
5. Distinguish from the sibling 'ACL policy not found' error: this one is a system fault, not a config mistake.

policies, _, _ := client.ACL().Policies().List(nil)
existing := map[string]bool{}
for _, p := range policies { existing[p.Name] = true }
for _, link := range role.Policies {
    if !existing[link.Name] { return fmt.Errorf("policy %q does not exist", link.Name) }
}

_, _, err := client.ACL().Roles().Create(role, nil)
if err != nil {
    if strings.Contains(err.Error(), "ACL policy not found") { /* config fix: create the policy */ }
    if strings.Contains(err.Error(), "ACL policy lookup failed") { /* system fault: retry/restart */ }
}

When it happens

Trigger: upsertACLRoleTxn (via UpsertACLRoles) when txn.FirstWatch("acl_policy", "id", policyLink.Name) returns a non-nil error — corrupted txn/table state, or patched code using a wrong index name.

Common situations: Corrupted server state store; embedded/modified Nomad where acl_policy index names changed; race with table rebuild in unusual deployments.

Related errors

• policy lookup failed: %v
• upserting policy failed: %v
• token lookup failed: %v
• acl token lookup failed: %v
• ACL role insert failed: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/005ee72e11d7f503. Report an issue: GitHub.