{"record":{"id":"37318c3cac98dece","repo":"hashicorp/nomad","slug":"failed-to-get-tainted-nodes-for-job-s-v","errorCode":null,"errorMessage":"failed to get tainted nodes for job '%s': %v","messagePattern":"failed to get tainted nodes for job '(.+?)': (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"scheduler/generic_sched.go","lineNumber":341,"sourceCode":"\t// Success!\n\treturn true, nil\n}\n\n// computeJobAllocs is used to reconcile differences between the job,\n// existing allocations and node status to update the allocations.\nfunc (s *GenericScheduler) computeJobAllocs() error {\n\t// Lookup the allocations by JobID\n\tws := memdb.NewWatchSet()\n\tallocs, err := s.state.AllocsByJob(ws, s.eval.Namespace, s.eval.JobID, true)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to get allocs for job '%s': %v\",\n\t\t\ts.eval.JobID, err)\n\t}\n\n\t// Determine the tainted nodes containing job allocs\n\ttainted, err := taintedNodes(s.state, allocs)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to get tainted nodes for job '%s': %v\",\n\t\t\ts.eval.JobID, err)\n\t}\n\n\t// Update the allocations which are in pending/running state on tainted\n\t// nodes to lost, but only if the scheduler has already marked them\n\tupdateNonTerminalAllocsToLost(s.plan, tainted, allocs)\n\n\tr := reconciler.NewAllocReconciler(s.logger,\n\t\tgenericAllocUpdateFn(s.ctx, s.stack, s.eval.ID),\n\t\treconciler.ReconcilerState{\n\t\t\tJob:               s.job,\n\t\t\tJobID:             s.eval.JobID,\n\t\t\tJobIsBatch:        s.batch,\n\t\t\tDeploymentCurrent: s.deployment,\n\t\t\tExistingAllocs:    allocs,\n\t\t\tEvalID:            s.eval.ID,\n\t\t\tEvalPriority:      s.eval.Priority,\n\t\t},","sourceCodeStart":323,"sourceCodeEnd":359,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/scheduler/generic_sched.go#L323-L359","documentation":"After fetching the job's allocations, computeJobAllocs calls taintedNodes to find nodes that are drained/down containing those allocs. If that state store query errors, processing aborts with this wrapped error.","triggerScenarios":"taintedNodes(s.state, allocs) returns an error — internally it does StateStore.NodesByIDs or node status lookups that fail due to state store/memdb errors during computeJobAllocs.","commonSituations":"Server state store instability especially with many tainted/drained nodes, raft restore, resource pressure.","solutions":["Inspect the wrapped underlying error for the root cause","Retry the eval; Nomad reprocesses failed evaluations","Verify node and raft health with 'nomad node status' and 'nomad operator raft list-peers'","Restore from a raft snapshot if state corruption is suspected"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"nomad node status // identify down/drained nodes and confirm cluster responsiveness","typeGuard":null,"tryCatchPattern":"// Inspect wrapped error in server logs; retry via eval reprocessing\nnomad node status -drained\nnomad eval list | grep -i failed","preventionTips":["Regularly clean up long-drained nodes","Monitor server state store health","Keep Nomad versions current","Maintain raft snapshots"],"tags":["nomad","scheduler","state-store","nodes"],"backgroundTag":"state-store-lookup-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}