hashicorp/nomad · error

unable to unmarshal ACLToken: %w

Error message

unable to unmarshal ACLToken: %w

What it means

decodeACLToken wraps a json.Unmarshal failure when decoding a base64-decoded Consul ACL token stored in the allocation's secure执行 directory. The token bytes exist but are not valid JSON matching consulapi.ACLToken, so the consul_hook cannot restore the alloc's Consul credentials. This is a defensive wrap so the underlying decode error (syntax, type mismatch, unexpected EOF) is preserved in the chain.

Source

Thrown at client/allocrunner/consul_hook.go:351

}

func newResourcesBackend(allocID string, hr *cstructs.AllocHookResources, db cstate.StateDB) *resourcesBackend {
	return &resourcesBackend{
		allocID:       allocID,
		hookResources: hr,
		db:            db,
	}
}

func decodeACLToken(b64ACLToken string, token *consulapi.ACLToken) error {
	decodedBytes, err := base64.StdEncoding.DecodeString(b64ACLToken)
	if err != nil {
		return fmt.Errorf("unable to process ACLToken: %w", err)
	}

	if len(decodedBytes) != 0 {
		if err := json.Unmarshal(decodedBytes, token); err != nil {
			return fmt.Errorf("unable to unmarshal ACLToken: %w", err)
		}
	}

	return nil
}

func encodeACLToken(token *consulapi.ACLToken) (string, error) {
	jsonBytes, err := json.Marshal(token)
	if err != nil {
		return "", fmt.Errorf("unable to marshal ACL token: %w", err)
	}

	return base64.StdEncoding.EncodeToString(jsonBytes), nil
}

// This function will never return nil, even in case of error
func (rs *resourcesBackend) loadAllocTokens() (map[string]map[string]*consulapi.ACLToken, error) {
	allocTokens := map[string]map[string]*consulapi.ACLToken{}

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Delete the stale/corrupt token file in the alloc dir and let Nomad re-fetch and re-persist the token from Consul on the next restart
2. Check the wrapped error (%w chain) to see the exact json.Unmarshal cause (e.g. invalid character, unexpected end of JSON input)
3. Ensure the token file was written by the same Nomad version; reschedule the allocation so it regenerates the file
4. Verify the base64 payload actually decodes to a consulapi.ACLToken JSON object, not a raw secret string

Example fix

// before: debugging with swallowed cause
return fmt.Errorf("unable to unmarshal ACLToken: %w", err)

// after: log/inspect the wrapped cause and recover
if err := json.Unmarshal(decodedBytes, token); err != nil {
	logger.Warn("corrupt persisted ACL token, re-creating", "err", err)
	os.Remove(tokenPath) // let the hook re-derive the token
	return fmt.Errorf("unable to unmarshal ACLToken: %w", err)
}

decoded, err := base64.StdEncoding.DecodeString(raw)
if err != nil { return err }
if len(decoded) == 0 || !json.Valid(decoded) {
	return fmt.Errorf("persisted ACL token is not valid JSON; removing stale file")
}
var probe map[string]any
if err := json.Unmarshal(decoded, &probe); err != nil {
	return fmt.Errorf("corrupt ACL token: %w", err)
}

func isValidACLTokenJSON(b []byte) bool {
	var t consulapi.ACLToken
	return json.Unmarshal(b, &t) == nil && t.SecretID != ""
}

if _, err := loadAllocTokens(...); err != nil {
	var jsonErr *json.UnmarshalTypeError
	if errors.As(err, &jsonErr) || strings.Contains(err.Error(), "unmarshal ACLToken") {
		// recover: delete corrupt token file and re-derive token
	}
}

When it happens

Trigger: loadAllocTokens reads the persisted token file from the alloc's token directory, base64-decodes it, and calls json.Unmarshal into *consulapi.ACLToken; unmarshal fails when the file is corrupt, truncated, hand-edited, or written by a different Nomad version with an incompatible schema.

Common situations: Disk corruption or partial writes on the client data dir; operator manually editing the token file; upgrading/downgrading Nomad so the stored token JSON no longer matches the struct; decoding a file that is base64 of something other than the marshaled token.

Understand the failure class

Background: "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it — this error's family across 23 libraries.

Related errors

• unable to marshal ACL token: %w
• no one-time token returned
• no ACL token returned
• errMissingACLRoleID
• errMissingACLAuthMethodName

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/0b015333a6aebc84. Report an issue: GitHub.