hashicorp/nomad · error
ACL role insert failed: %v
Error message
ACL role insert failed: %v
What it means
Wraps an error returned by txn.Insert into the acl_roles table during upsertACLRoleTxn. The memdb insert failed for a low-level reason (invalid object, table/txn state), after uniqueness and policy-link validation already passed. The raft transaction aborts and no role is written.
Source
Thrown at nomad/state/state_store_acl.go:158
if existing != nil {
// If the role already exists, check whether the update contains any
// difference. If it doesn't, we can avoid a state update as wel as
// updates to any blocking queries.
if existing.Equal(role) {
return false, nil
}
role.CreateIndex = existing.CreateIndex
role.ModifyIndex = index
} else {
role.CreateIndex = index
role.ModifyIndex = index
}
// Insert the role into the table.
if err := txn.Insert(TableACLRoles, role); err != nil {
return false, fmt.Errorf("ACL role insert failed: %v", err)
}
return true, nil
}
// validateACLRolePolicyLinksTxn is the same as ValidateACLRolePolicyLinks but
// allows callers to pass their own transaction.
func (s *StateStore) validateACLRolePolicyLinksTxn(txn *txn, role *structs.ACLRole) error {
for _, policyLink := range role.Policies {
_, existing, err := txn.FirstWatch("acl_policy", indexID, policyLink.Name)
if err != nil {
return fmt.Errorf("ACL policy lookup failed: %v", err)
}
if existing == nil {
return errors.New("ACL policy not found")
}
}
return nil
}View on GitHub (pinned to 482b49bf1a)
Solutions
- Retry the operation; if via RPC, the CLI will surface the wrapped memdb message — retry after confirming leader stability.
- Restart the server to rebuild the state store from raft.
- Validate the ACLRole payload (ID, Name, Policies) conforms to the current structs.ACLRole schema for your Nomad version.
- Check for Nomad version mismatches between server and any direct state-store manipulation (debug tooling).
- File an upstream issue with the full wrapped error if reproducible.
Defensive patterns
Strategy: retry
Validate before calling
// Ensure the payload matches the SDK schema before sending.
if role.Name == "" || role.ID == "" || len(role.Policies) == 0 { return errors.New("invalid ACLRole payload") } Try / catch
_, _, err := client.ACL().Roles().Update(role, nil)
if err != nil && strings.Contains(err.Error(), "ACL role insert failed") {
// retry with backoff; escalate to server restart if persistent
} Prevention
- Use the official API client to construct ACLRole objects (schema-safe).
- Keep servers on stock, supported Nomad versions.
- Retry idempotent raft operations with backoff.
- Watch for state-store corruption indicators in logs.
When it happens
Trigger: UpsertACLRoles / applyACLRolesDelete path when txn.Insert(TableACLRoles, role) errors — typically a malformed *structs.ACLRole (violating schema expectations) or a broken/expired transaction in an embedded or patched state store.
Common situations: Custom Nomad forks or enterprise patches that construct ACLRole objects incorrectly; corrupted memdb after a crash; test harnesses inserting invalid fixtures.
Related errors
- policy lookup failed: %v
- upserting policy failed: %v
- token lookup failed: %v
- acl token lookup failed: %v
- ACL policy lookup failed: %v
AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04).
Data as JSON: /api/errors/8c57a9df2a7239fe.
Report an issue: GitHub.