{"record":{"id":"861a52f161697e53","repo":"hashicorp/nomad","slug":"failed-to-look-up-job-versions-for-q-v","errorCode":null,"errorMessage":"failed to look up job versions for %q: %v","messagePattern":"failed to look up job versions for %q: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/state/state_store.go","lineNumber":2255,"sourceCode":"\t// JobTrackedVersions really must not be zero here\n\tif err := s.config.Validate(); err != nil {\n\t\treturn err\n\t}\n\n\t// Insert the job\n\tif err := txn.Insert(\"job_version\", job); err != nil {\n\t\treturn fmt.Errorf(\"failed to insert job into job_version table: %v\", err)\n\t}\n\n\tif err := txn.Insert(\"index\", &IndexEntry{\"job_version\", index}); err != nil {\n\t\treturn fmt.Errorf(\"index update failed: %v\", err)\n\t}\n\n\t// Get all the historic jobs for this ID, except those with a VersionTag,\n\t// as they should always be kept. They are in Version order, high to low.\n\tall, err := s.jobVersionByID(txn, nil, job.Namespace, job.ID, false)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to look up job versions for %q: %v\", job.ID, err)\n\t}\n\n\t// If we are below the limit there is no GCing to be done\n\tif len(all) <= s.config.JobTrackedVersions {\n\t\treturn nil\n\t}\n\n\t// We have to delete a historic job to make room.\n\t// Find index of the highest versioned stable job\n\tstableIdx := -1\n\tfor i, j := range all {\n\t\tif j.Stable {\n\t\t\tstableIdx = i\n\t\t\tbreak\n\t\t}\n\t}\n\n\t// If the stable job is the oldest version, do a swap to bring it into the","sourceCodeStart":2237,"sourceCodeEnd":2273,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/state/state_store.go#L2237-L2273","documentation":"This error is wrapped inside the job-version garbage collection logic in Nomad's state store. When the number of historic versions of a job exceeds JobTrackedVersions, the store GCs the oldest versions; if reading all versions of the job ID via jobVersionByID fails, this error is returned wrapping the underlying cause. It means the internal job_version table could not be iterated, typically a memdb/index-level failure.","triggerScenarios":"Calling job registration/delete paths that trigger version trimming (deleteJobVersion) when jobVersionByID fails to look up versions for the job ID in the given namespace, e.g. corrupted txn index or internal table read error.","commonSituations":"Rare internal state-store failures: memdb index errors, transaction invalidation, or bugs during job updates on clusters with many job versions (JobTrackedVersions exceeded).","solutions":["Inspect the wrapped %v cause for the underlying memdb/txn error","Retry the job registration; state store ops are transactional and transient errors rarely persist","If persistent, capture a debug bundle and file with Nomad; may indicate state corruption","Check JobTrackedVersions config and job churn; reduce version churn to avoid trimming paths"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// pre-check version count via API\nstatus, _ := client.Jobs().Info(jobID)\n_ = status // ensure job exists and churn is low","typeGuard":null,"tryCatchPattern":"// retry transient state store errors\nfor i := 0; i < 3; i++ {\n  _, err := client.Jobs().Register(job, nil)\n  if err == nil || !strings.Contains(err.Error(), \"failed to look up job versions\") { break }\n  time.Sleep(backoff)\n}","preventionTips":["Keep JobTrackedVersions appropriate for churn","Retry job registrations on transient errors","Monitor server logs for repeated state store failures"],"tags":["nomad","state-store","job-versions","gc"],"backgroundTag":"state-store-read-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}