{"record":{"id":"94d44346aeaf0696","repo":"hashicorp/nomad","slug":"consul-tokens-for-cluster-s-requested-by-task-s","errorCode":null,"errorMessage":"consul tokens for cluster %s requested by task %s not found","messagePattern":"consul tokens for cluster (.+?) requested by task (.+?) not found","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/allocrunner/taskrunner/template_hook.go","lineNumber":158,"sourceCode":"\tfor _, wid := range req.Task.Identities {\n\t\tif wid.Name == consulWIDName {\n\t\t\thasConsulIdentity = true\n\t\t\tbreak\n\t\t}\n\t}\n\n\t// If we leave the Consul token as an empty string, then consul-template\n\t// will try to pick it up from the environment; we want to enforce that we\n\t// don't have a Consul token unless intentionally configured\n\th.consulToken = \"invalid-token\"\n\n\tif hasConsulIdentity {\n\t\tconsulCluster := req.Task.GetConsulClusterName(tg)\n\t\tconsulTokens := h.config.hookResources.GetConsulTokens()\n\t\tclusterTokens := consulTokens[consulCluster]\n\n\t\tif clusterTokens == nil {\n\t\t\treturn fmt.Errorf(\n\t\t\t\t\"consul tokens for cluster %s requested by task %s not found\",\n\t\t\t\tconsulCluster, req.Task.Name,\n\t\t\t)\n\t\t}\n\n\t\tconsulToken := clusterTokens[consulWIDName+\"/\"+req.Task.Name]\n\t\tif consulToken == nil {\n\t\t\treturn fmt.Errorf(\n\t\t\t\t\"consul tokens for cluster %s and identity %s requested by task %s not found\",\n\t\t\t\tconsulCluster, consulWIDName, req.Task.Name,\n\t\t\t)\n\t\t}\n\n\t\th.consulToken = consulToken.SecretID\n\t} else if h.config.clientConfig.TemplateConfig != nil &&\n\t\th.config.clientConfig.TemplateConfig.UseClientConsulToken {\n\t\tconsulCluster := req.Task.GetConsulClusterName(tg)\n\t\tif config, ok := h.config.clientConfig.ConsulConfigs[consulCluster]; ok {","sourceCodeStart":140,"sourceCodeEnd":176,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/allocrunner/taskrunner/template_hook.go#L140-L176","documentation":"The template hook requires Consul workload identity tokens to be pre-populated in the allocation's hook resources before Prestart runs. When the task requests a Consul identity but the token map for the resolved Consul cluster is empty/nil, Prestart fails with this error. It indicates the token derivation step never delivered tokens for that cluster.","triggerScenarios":"A task defines a consul identity (hasConsulIdentity true) but h.config.hookResources.GetConsulTokens() returns a map with no entry for the cluster name returned by req.Task.GetConsulClusterName(tg) — e.g. the Consul token for that cluster was never derived or was dropped during alloc restore.","commonSituations":"Mistyped consul cluster name in the job's consul block vs the client's consul config; upgrading Nomad to workload identities without the token derivation path running; alloc restore losing hook resources.","solutions":["Verify the task's consul.cluster block matches a cluster configured on the client agent","Ensure the Nomad server has Consul identities enabled and tokens can be derived (check Consul login/policy setup)","Rerun the allocation so Prestart token derivation executes again","Check nomad client logs for upstream token derivation errors"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// fail fast before Prestart depends on derived tokens\nconsulTokens := hookResources.GetConsulTokens()\nclusterTokens, ok := consulTokens[task.GetConsulClusterName(tg)]\nif !ok || clusterTokens == nil {\n    return fmt.Errorf(\"consul tokens missing for cluster; check consul.cluster in job vs client config\")\n}","typeGuard":null,"tryCatchPattern":"if clusterTokens == nil {\n    return fmt.Errorf(\"consul tokens for cluster %q not derived — verify server-side consul identity config and redeploy the job\", consulCluster)\n}","preventionTips":["Keep the job's consul cluster name in sync with the client agent's consul stanza names","Use consistent identity naming for tasks (identity-name/task-name key format)","After upgrading Nomad, verify workload identity token derivation is enabled on servers","Watch client logs for token derivation errors before allocs start"],"tags":["consul","workload-identity","nomad-client","tokens"],"backgroundTag":"consul-token-not-found","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}