hashicorp/nomad · error

consul tokens for cluster %s requested by task %s not found

Error message

consul tokens for cluster %s requested by task %s not found

What it means

The template hook requires Consul workload identity tokens to be pre-populated in the allocation's hook resources before Prestart runs. When the task requests a Consul identity but the token map for the resolved Consul cluster is empty/nil, Prestart fails with this error. It indicates the token derivation step never delivered tokens for that cluster.

Source

Thrown at client/allocrunner/taskrunner/template_hook.go:158

	for _, wid := range req.Task.Identities {
		if wid.Name == consulWIDName {
			hasConsulIdentity = true
			break
		}
	}

	// If we leave the Consul token as an empty string, then consul-template
	// will try to pick it up from the environment; we want to enforce that we
	// don't have a Consul token unless intentionally configured
	h.consulToken = "invalid-token"

	if hasConsulIdentity {
		consulCluster := req.Task.GetConsulClusterName(tg)
		consulTokens := h.config.hookResources.GetConsulTokens()
		clusterTokens := consulTokens[consulCluster]

		if clusterTokens == nil {
			return fmt.Errorf(
				"consul tokens for cluster %s requested by task %s not found",
				consulCluster, req.Task.Name,
			)
		}

		consulToken := clusterTokens[consulWIDName+"/"+req.Task.Name]
		if consulToken == nil {
			return fmt.Errorf(
				"consul tokens for cluster %s and identity %s requested by task %s not found",
				consulCluster, consulWIDName, req.Task.Name,
			)
		}

		h.consulToken = consulToken.SecretID
	} else if h.config.clientConfig.TemplateConfig != nil &&
		h.config.clientConfig.TemplateConfig.UseClientConsulToken {
		consulCluster := req.Task.GetConsulClusterName(tg)
		if config, ok := h.config.clientConfig.ConsulConfigs[consulCluster]; ok {

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Verify the task's consul.cluster block matches a cluster configured on the client agent
2. Ensure the Nomad server has Consul identities enabled and tokens can be derived (check Consul login/policy setup)
3. Rerun the allocation so Prestart token derivation executes again
4. Check nomad client logs for upstream token derivation errors

// fail fast before Prestart depends on derived tokens
consulTokens := hookResources.GetConsulTokens()
clusterTokens, ok := consulTokens[task.GetConsulClusterName(tg)]
if !ok || clusterTokens == nil {
    return fmt.Errorf("consul tokens missing for cluster; check consul.cluster in job vs client config")
}

if clusterTokens == nil {
    return fmt.Errorf("consul tokens for cluster %q not derived — verify server-side consul identity config and redeploy the job", consulCluster)
}

When it happens

Trigger: A task defines a consul identity (hasConsulIdentity true) but h.config.hookResources.GetConsulTokens() returns a map with no entry for the cluster name returned by req.Task.GetConsulClusterName(tg) — e.g. the Consul token for that cluster was never derived or was dropped during alloc restore.

Common situations: Mistyped consul cluster name in the job's consul block vs the client's consul config; upgrading Nomad to workload identities without the token derivation path running; alloc restore losing hook resources.

Related errors

• consul tokens for cluster %s and identity %s requested by ta
• error getting signed identity for task %s: %v
• error getting signed identity for service %s: %v
• failed to write Consul SI token: %w
• No client configuration found for Vault cluster %s

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/94d44346aeaf0696. Report an issue: GitHub.