{"record":{"id":"fb001a2308376b94","repo":"hashicorp/nomad","slug":"allocation-is-terminal","errorCode":null,"errorMessage":"allocation is terminal","messagePattern":"allocation is terminal","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/auth/auth.go","lineNumber":734,"sourceCode":"\treturn resolveAuthorizedClientNodePoolByNodeID(snap, aclObj, registration.NodeID)\n}\n\nfunc (s *Authenticator) verifyWorkloadIdentityClaim(claims *structs.IdentityClaims) error {\n\tsnap, err := s.getState().Snapshot()\n\tif err != nil {\n\t\treturn err\n\t}\n\talloc, err := snap.AllocByID(nil, claims.AllocationID)\n\tif err != nil {\n\t\treturn err\n\t}\n\tif alloc == nil || alloc.Job == nil {\n\t\treturn fmt.Errorf(\"allocation does not exist\")\n\t}\n\n\t// the claims for terminal allocs are always treated as expired\n\tif alloc.ClientTerminalStatus() {\n\t\treturn fmt.Errorf(\"allocation is terminal\")\n\t}\n\n\treturn nil\n}\n\nfunc (s *Authenticator) resolveClaims(claims *structs.IdentityClaims) (*acl.ACL, error) {\n\n\t// Nomad node identity claims currently map to a client ACL. If we open this\n\t// up in the future, we will want to modify this section to perform similar\n\t// work that is done for workload claims.\n\tif claims.IsNode() {\n\t\tif claims.NodeIdentityClaims == nil || claims.NodeIdentityClaims.NodePool == \"\" {\n\t\t\treturn nil, fmt.Errorf(\"node identity claims missing node pool\")\n\t\t}\n\t\treturn acl.NewClientACL(claims.NodeIdentityClaims.NodePool), nil\n\t}\n\n\tpolicies, err := s.ResolvePoliciesForClaims(claims)","sourceCodeStart":716,"sourceCodeEnd":752,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/auth/auth.go#L716-L752","documentation":"verifyWorkloadIdentityClaim in nomad/auth/auth.go rejects workload-identity claims whose allocation is in a terminal client state (complete, failed, lost). Terminal allocations no longer run workloads, so signing/verifying new claims for them is treated as an expired claim. This prevents stale or re-played alloc claims from gaining credentials after the workload has stopped.","triggerScenarios":"Calling VerifyClaim (e.g. via the ACS/Workload Auth RPC) with a claim whose AllocationID resolves to an allocation whose ClientTerminalStatus() is true, i.e. the client reported the alloc as complete, failed, or lost.","commonSituations":"A client retries claim verification after the allocation finished; a lost node's allocs are marked lost by the server while the client still holds a valid-looking JWT; clock/retry issues cause verification long after job completion; drivers restart and re-authenticate dead allocs.","solutions":["Check the allocation's client status before verifying; if it is terminal (complete/failed/lost), do not request new credentials for it.","Re-run the workload so a fresh, non-terminal allocation is created and use its allocation ID/claims.","If the alloc looks terminal but should be running, check node health and reschedule; a 'lost' status usually means the node missed heartbeats."],"exampleFix":"// before\nclaims, err := auth.VerifyClaim(ctx, req) // fails: \"allocation is terminal\"\n// after\nalloc, _ := client_allocs.GetAlloc(req.AllocationID)\nif alloc != nil && !alloc.ClientTerminalStatus() {\n    claims, err = auth.VerifyClaim(ctx, req)\n} else {\n    // reschedule or skip credential refresh for this alloc\n}","handlingStrategy":"validation","validationCode":"alloc, _, err := client.Allocs().Info(allocID, nil)\nif err != nil { return err }\nif alloc.ClientStatus == \"complete\" || alloc.ClientStatus == \"failed\" || alloc.ClientStatus == \"lost\" {\n    return fmt.Errorf(\"alloc %s is terminal; cannot verify claim\", allocID)\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Only refresh workload credentials for allocations in a running/pending client state.","Handle claim errors by rescheduling, not retrying, for terminal allocs.","Monitor node heartbeats so allocs don't get marked lost while clients still run."],"tags":["nomad","workload-identity","authentication","allocation"],"backgroundTag":"workload-identity-claim-rejected","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}