detected corrupted token within the state store: missing rol

Error message

detected corrupted token within the state store: missing role link ID

What it means

fixTokenRoleLinks repairs ACL tokens whose role links may reference deleted or invalid roles. This error fires when a token's role link has an empty ID — data that should never exist — so the store returns a corruption error instead of panicking or dereferencing an empty key. It is returned by token lookups (ACLTokenByAccessorID/SecretID) while sanitizing the token.

Source

Thrown at nomad/state/state_store_acl.go:323

	// operating on the token directly from state.
	copied := false

	token := original

	// copyTokenFn is a helper function which copies the ACL token along with
	// a certain number of ACL role links.
	copyTokenFn := func(t *structs.ACLToken, numLinks int) *structs.ACLToken {
		clone := t.Copy()
		clone.Roles = slices.Clone(t.Roles[:numLinks])
		return clone
	}

	for linkIndex, link := range original.Roles {

		// This should never happen, but guard against it anyway, so we log an
		// error rather than panic.
		if link.ID == "" {
			return nil, errors.New("detected corrupted token within the state store: missing role link ID")
		}

		role, err := s.getACLRoleByIDTxn(txn, nil, link.ID)
		if err != nil {
			return nil, err
		}

		if role == nil {
			if !copied {
				// clone the token as we cannot touch the original
				token = copyTokenFn(original, linkIndex)
				copied = true
			}
			// if already owned then we just don't append it.
		} else if role.Name != link.Name {
			if !copied {
				token = copyTokenFn(original, linkIndex)
				copied = true

View on GitHub (pinned to 482b49bf1a)

Solutions

Identify the corrupt token (accessor ID from the log path) and delete/recreate it: nomad acl token delete -_accessor <id>.
Restore state from a known-good snapshot taken before the corruption.
Upgrade Nomad to the latest patch release; this guard was added for a known corruption class.
If many tokens are affected, contact HashiCorp support and do not hand-edit raft data.

Defensive patterns

Strategy: retry

Try / catch

tok, _, err := client.ACLTokens().GetAccessor(accessorID, nil)
if err != nil && strings.Contains(err.Error(), "corrupted token") {
    // do not retry blindly: escalate to operator, then delete the token
    return escalateAndDeleteToken(accessorID)
}

Prevention

Keep regular `nomad operator snapshot save` backups of the state store.
Never hand-edit raft/state data.
Upgrade Nomad promptly when state-store corruption patches are released.
Monitor for this error in server logs as a critical alert.

When it happens

Trigger: Reading any ACL token via accessor or secret ID when the stored token record contains a Roles[] entry with an empty ID string; typically the result of corrupted state store data, a bad raft snapshot restore, or a bug in older token-write code paths.

Common situations: Upgrading from a version where token role links were mishandled; restoring a state snapshot from an unhealthy cluster; manual raft/state manipulation.

Related errors

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/9179acae5eb7b800. Report an issue: GitHub.