{"record":{"id":"851ac3ef9718b5be","repo":"hashicorp/nomad","slug":"jobbyid-lookup-failed-w","errorCode":null,"errorMessage":"JobByID lookup failed: %w","messagePattern":"JobByID lookup failed: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/fsm.go","lineNumber":913,"sourceCode":"\t}\n\n\tif purge {\n\t\tif err := n.state.DeleteJobTxn(index, namespace, jobID, tx); err != nil {\n\t\t\treturn fmt.Errorf(\"DeleteJob failed: %w\", err)\n\t\t}\n\n\t\t// We always delete from the periodic launch table because it is possible that\n\t\t// the job was updated to be non-periodic, thus checking if it is periodic\n\t\t// doesn't ensure we clean it up properly.\n\t\tn.state.DeletePeriodicLaunchTxn(index, namespace, jobID, tx)\n\t\treturn nil\n\t}\n\n\t// Get the current job and mark it as stopped and re-insert it.\n\tws := memdb.NewWatchSet()\n\tcurrent, err := n.state.JobByIDTxn(ws, namespace, jobID, tx)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"JobByID lookup failed: %w\", err)\n\t}\n\n\tif current == nil {\n\t\treturn fmt.Errorf(\"job %q in namespace %q doesn't exist to be deregistered\", jobID, namespace)\n\t}\n\n\tstopped := current.Copy()\n\tstopped.Stop = true\n\tif submitTime != 0 {\n\t\tstopped.SubmitTime = submitTime\n\t}\n\n\t// Disable scaling policies to avoid monitoring stopped jobs\n\tscalingPolicies := stopped.GetScalingPolicies()\n\tfor _, policy := range scalingPolicies {\n\t\tpolicy.Enabled = false\n\t}\n","sourceCodeStart":895,"sourceCodeEnd":931,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/fsm.go#L895-L931","documentation":"After removing periodic tracking, handleJobDeregister fetches the current job record with state.JobByIDTxn to mark it stopped and re-insert it. An error from that lookup (distinct from a nil result) is wrapped as \"JobByID lookup failed\". This is a state-store read failure, not a missing job.","triggerScenarios":"JobByIDTxn returns a non-nil error during FSM apply of a deregistration — internal memdb iterator error, transaction abort, or memory failure.","commonSituations":"Server under memory pressure, corrupted in-memory state store, or txn issues after a failed prior operation in the same apply.","solutions":["Read the wrapped underlying error to identify the memdb/state-store cause.","Restart the server agent to rebuild state from Raft.","Check server resource limits (RAM) and host logs for OOM events.","If reproducible, capture a goroutine/profile dump and report to Nomad."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"err := client.Jobs().Deregister(jobID, false, nil)\nif err != nil && strings.Contains(err.Error(), \"JobByID lookup failed\") {\n    // transient server-side error: check server health, retry\n}","preventionTips":["Keep servers resourced to avoid memdb failures","Maintain same-version server fleet","Monitor server agent logs for state-store errors"],"tags":["nomad","raft","fsm","state-store"],"backgroundTag":"job-lookup-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}