{"record":{"id":"82e2825fd26539a7","repo":"hashicorp/nomad","slug":"state-for-allocation-s-not-found-on-client","errorCode":null,"errorMessage":"state for allocation %s not found on client","messagePattern":"state for allocation (.+?) not found on client","errorType":"http","errorClass":null,"httpStatus":404,"severity":"error","filePath":"client/fs_endpoint.go","lineNumber":187,"sourceCode":"\n\tif err := decoder.Decode(&req); err != nil {\n\t\thandleStreamResultError(err, new(int64(http.StatusInternalServerError)), encoder)\n\t\treturn\n\t}\n\n\tif req.AllocID == \"\" {\n\t\thandleStreamResultError(allocIDNotPresentErr, new(int64(http.StatusBadRequest)), encoder)\n\t\treturn\n\t}\n\n\tar, err := f.c.getAllocRunner(req.AllocID)\n\tif err != nil {\n\t\thandleStreamResultError(structs.NewErrUnknownAllocation(req.AllocID), new(int64(http.StatusNotFound)), encoder)\n\t\treturn\n\t}\n\tif ar.IsDestroyed() {\n\t\thandleStreamResultError(\n\t\t\tfmt.Errorf(\"state for allocation %s not found on client\", req.AllocID),\n\t\t\tnew(int64(http.StatusNotFound)),\n\t\t\tencoder,\n\t\t)\n\t\treturn\n\t}\n\talloc := ar.Alloc()\n\n\t// Check read permissions\n\tif aclObj, err := f.c.ResolveToken(req.QueryOptions.AuthToken); err != nil {\n\t\thandleStreamResultError(err, new(int64(http.StatusForbidden)), encoder)\n\t\treturn\n\t} else if !aclObj.AllowNsOp(alloc.Namespace, acl.NamespaceCapabilityReadFS) {\n\t\thandleStreamResultError(structs.ErrPermissionDenied, new(int64(http.StatusForbidden)), encoder)\n\t\treturn\n\t}\n\n\t// Validate the arguments\n\tif req.Path == \"\" {","sourceCodeStart":169,"sourceCodeEnd":205,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/fs_endpoint.go#L169-L205","documentation":"Nomad's client filesystem streaming endpoint (HTTP GET /v1/client/fs/stream) rejects the request when the allocation's state has been destroyed on the client node. The AllocRunner's state exists but IsDestroyed() reports true, meaning the allocation has finished shutting down and its resources were reclaimed. The server returns HTTP 404 to the caller.","triggerScenarios":"Calling the file stream API for an allocation whose AllocRunner on the target client node has been destroyed — typically after the alloc completed, was stopped, or was garbage collected on that node while a stream request was in flight or arrived just after destruction.","commonSituations":"Streaming logs/files from an allocation that just finished its task; a race where the job was stopped moments before the fs/stream request; stale alloc IDs cached in tooling pointing at a GC'd allocation; client node restarts where destroyed alloc state lingers as a tombstone.","solutions":["Query the allocation's current status via the API and confirm it is running before streaming files.","Handle HTTP 404 by refreshing the alloc ID (e.g., look up the replacement allocation after a reschedule) and retrying against the new alloc.","Verify you are targeting the correct client node; a destroyed alloc may exist as a tombstone on one node while its replacement runs on another (use ?node_id= to pick the right node).","If this occurs during shutdown races, retry with backoff or treat it as a terminal 'alloc gone' condition rather than a transient failure."],"exampleFix":"// before\nstream, err := client.Allocs().Logs(alloc, trace, \"web\", \"stdout\", nil, nil)\n// after\nalloc, _, err := client.Allocations().Info(alloc.ID, nil)\nif err != nil || alloc.ClientStatus != \"running\" {\n    return fmt.Errorf(\"allocation %s is not running on client\", allocID)\n}\nstream, err = client.Allocs().Logs(alloc, trace, \"web\", \"stdout\", nil, nil)","handlingStrategy":"validation","validationCode":"a, _, err := client.Allocations().Info(allocID, nil)\nif err != nil || a.ClientStatus != \"running\" {\n    return fmt.Errorf(\"alloc %s not running\", allocID)\n}","typeGuard":"func isAllocGone(err error) bool {\n    return strings.Contains(err.Error(), \"not found on client\") ||\n        strings.Contains(err.Error(), \"unknown allocation\")\n}","tryCatchPattern":"if err := stream(); err != nil {\n    if isAllocGone(err) {\n        alloc = reResolveAllocation(allocID) // handle reschedule\n        return\n    }\n    return err\n}","preventionTips":["Check alloc ClientStatus before streaming files.","Re-resolve alloc IDs after job stop/reschedule events.","Target the node explicitly when multiple nodes may know the alloc."],"tags":["nomad","client","allocation","http-404","streaming"],"backgroundTag":"allocation-not-found","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}