{"record":{"id":"5eebe86ea8c7a68a","repo":"hashicorp/nomad","slug":"failed-to-lookup-job-deployments-v","errorCode":null,"errorMessage":"failed to lookup job deployments: %v","messagePattern":"failed to lookup job deployments: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"scheduler/generic_sched.go","lineNumber":456,"sourceCode":"\n\tdestructive := make([]reconciler.PlacementResult, 0, len(result.DestructiveUpdate))\n\tfor _, p := range result.DestructiveUpdate {\n\t\ts.queuedAllocs[p.TaskGroup().Name] += 1\n\t\tdestructive = append(destructive, p)\n\t}\n\treturn s.computePlacements(destructive, place, result.TaskGroupAllocNameIndexes)\n}\n\n// downgradedJobForPlacement returns the previous stable version of the job for\n// downgrading a placement for non-canaries\nfunc (s *GenericScheduler) downgradedJobForPlacement(p reconciler.PlacementResult) (string, *structs.Job, error) {\n\tns, jobID := s.job.Namespace, s.job.ID\n\ttgName := p.TaskGroup().Name\n\n\t// find deployments and use the latest promoted or canaried version\n\tdeployments, err := s.state.DeploymentsByJobID(nil, ns, jobID, false)\n\tif err != nil {\n\t\treturn \"\", nil, fmt.Errorf(\"failed to lookup job deployments: %v\", err)\n\t}\n\n\tsort.Slice(deployments, func(i, j int) bool {\n\t\treturn deployments[i].JobVersion > deployments[j].JobVersion\n\t})\n\n\tfor _, d := range deployments {\n\t\t// It's unexpected to have a recent deployment that doesn't contain the TaskGroup; as all allocations\n\t\t// should be destroyed. In such cases, attempt to find the deployment for that TaskGroup and hopefully\n\t\t// we will kill it soon.  This is a defensive measure, have not seen it in practice\n\t\t//\n\t\t// Zero dstate.DesiredCanaries indicates that the TaskGroup allocates were updated in-place without using canaries.\n\t\tif dstate := d.TaskGroups[tgName]; dstate != nil && (dstate.Promoted || dstate.DesiredCanaries == 0) {\n\t\t\tjob, err := s.state.JobByIDAndVersion(nil, ns, jobID, d.JobVersion)\n\t\t\treturn d.ID, job, err\n\t\t}\n\t}\n","sourceCodeStart":438,"sourceCodeEnd":474,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/scheduler/generic_sched.go#L438-L474","documentation":"downgradedJobForPlacement (used for disconnected-client / lost-restore handling) looks up the job's deployments via StateStore.DeploymentsByJobID to find the latest promoted or canaried version. A state store error aborts the placement with this wrapped error.","triggerScenarios":"StateStore.DeploymentsByJobID(nil, ns, jobID, false) errors during computePlacements when a placement needs a downgraded job version (client reconnected after disconnection).","commonSituations":"State store instability while handling disconnected clients rejoining, heavy deployment churn, server resource pressure.","solutions":["Check the wrapped underlying error in server logs","Retry — the eval will be rescheduled","Verify state store and raft health; restore from snapshot if needed","Upgrade Nomad if the error recurs under disconnected-client scenarios"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"nomad deployment list // confirm deployment records are queryable before relying on reconnect flows","typeGuard":null,"tryCatchPattern":"// Inspect the wrapped error in server logs; rely on Nomad's eval retry\nnomad deployment list\nnomad eval list | grep -i failed","preventionTips":["Monitor server health during client disconnect/reconnect storms","Upgrade Nomad for disconnected-client fixes","Keep deployment history pruned to reasonable sizes","Maintain raft snapshots"],"tags":["nomad","scheduler","deployment","state-store"],"backgroundTag":"state-store-lookup-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}