{"record":{"id":"35038058d6b3b976","repo":"hashicorp/nomad","slug":"deployment-q-references-unknown-job-q","errorCode":null,"errorMessage":"deployment %q references unknown job %q","messagePattern":"deployment %q references unknown job %q","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/deploymentwatcher/deployments_watcher.go","lineNumber":300,"sourceCode":"\n\t// Already watched so just update the deployment\n\tif w, ok := w.watchers[d.ID]; ok {\n\t\tw.updateDeployment(d)\n\t\treturn nil, nil\n\t}\n\n\t// Get the job the deployment is referencing\n\tsnap, err := w.state.Snapshot()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tjob, err := snap.JobByID(nil, d.Namespace, d.JobID)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif job == nil {\n\t\treturn nil, fmt.Errorf(\"deployment %q references unknown job %q\", d.ID, d.JobID)\n\t}\n\n\twatcher := newDeploymentWatcher(w.ctx, w.queryLimiter, w.logger, w.state, d, job,\n\t\tw, w.deploymentRPC, w.jobRPC)\n\tw.watchers[d.ID] = watcher\n\treturn watcher, nil\n}\n\n// remove stops watching a deployment. This can be because the deployment is\n// complete or being deleted.\nfunc (w *Watcher) remove(d *structs.Deployment) {\n\tw.l.Lock()\n\tdefer w.l.Unlock()\n\tw.removeByIDLocked(d.ID)\n}\n\nfunc (w *Watcher) removeByIDLocked(id string) {\n\t// Not enabled so no-op","sourceCodeStart":282,"sourceCodeEnd":318,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/deploymentwatcher/deployments_watcher.go#L282-L318","documentation":"addLocked validates that the deployment's job still exists before creating its watcher. Nomad throws this when the deployment record points to a job that is no longer in the state store (e.g. the job was purged/GC'd while the deployment row remains).","triggerScenarios":"add()/forceAdd() -> addLocked: snap.JobByID(nil, d.Namespace, d.JobID) returns nil for the deployment's namespace/job ID.","commonSituations":"Job deregistration followed by a deployment re-add after leader failover; namespace-scoped lookups where the job exists in a different namespace than queried; GC removing jobs faster than deployments.","solutions":["Re-register the job (nomad job run) so the deployment's JobID resolves again","Verify the job exists in the deployment's namespace, not the default one","Cancel the orphaned deployment (nomad deployment fail) since it can no longer progress","Check that job GC isn't purging jobs while deployments still reference them"],"exampleFix":"// before: assuming default namespace\njob, _ := snap.JobByID(nil, d.Namespace, d.JobID)\n// after: ensure the job is registered in the deployment's namespace before re-adding\nif job == nil { cancel deployment or re-register: nomad job run -namespace <d.Namespace> <d.JobID> }\n","handlingStrategy":"validation","validationCode":"job, err := state.JobByID(nil, dep.Namespace, dep.JobID)\nif err == nil && job != nil {\n    watcher.Add(dep) // safe\n}","typeGuard":null,"tryCatchPattern":"try {\n    watcher.add(dep)\n} catch (e) {\n    if (e.message.includes('references unknown job')) {\n        // job was GC'd/purged; fail the deployment and re-register the job\n        failDeployment(dep.ID)\n        registerJob(dep.Namespace, dep.JobID)\n    } else { throw e }\n}","preventionTips":["Keep jobs registered while their deployments are active","Always query jobs with the deployment's namespace, not a hardcoded one","Tune GC so jobs aren't purged while deployments still reference them"],"tags":["nomad","deployment","job","state-store"],"backgroundTag":"referenced-job-not-found","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}