{"record":{"id":"cc26972a2c0fd98b","repo":"hashicorp/nomad","slug":"unable-to-upsert-job-into-job-version-table-v","errorCode":null,"errorMessage":"unable to upsert job into job_version table: %v","messagePattern":"unable to upsert job into job_version table: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/state/state_store.go","lineNumber":1888,"sourceCode":"\t\t\treturn fmt.Errorf(\"setting job status for %q failed: %v\", job.ID, err)\n\t\t}\n\n\t\t// Have to get the job again since it could have been updated\n\t\tupdated, err := txn.First(\"jobs\", \"id\", job.Namespace, job.ID)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"job lookup failed: %v\", err)\n\t\t}\n\t\tif updated != nil {\n\t\t\tjob = updated.(*structs.Job)\n\t\t}\n\t}\n\n\tif err := s.updateSummaryWithJob(index, job, txn); err != nil {\n\t\treturn fmt.Errorf(\"unable to create job summary: %v\", err)\n\t}\n\n\tif err := s.upsertJobVersion(index, job, txn); err != nil {\n\t\treturn fmt.Errorf(\"unable to upsert job into job_version table: %v\", err)\n\t}\n\n\tif err := s.updateJobScalingPolicies(index, job, txn); err != nil {\n\t\treturn fmt.Errorf(\"unable to update job scaling policies: %v\", err)\n\t}\n\n\tif err := s.updateJobRecommendations(index, txn, existingJob, job); err != nil {\n\t\treturn fmt.Errorf(\"unable to update job recommendations: %v\", err)\n\t}\n\n\tif err := s.updateJobCSIPlugins(index, job, existingJob, txn); err != nil {\n\t\treturn fmt.Errorf(\"unable to update job csi plugins: %v\", err)\n\t}\n\n\tif err := s.updateJobSubmission(index, sub, job.Namespace, job.ID, job.Version, txn); err != nil {\n\t\treturn fmt.Errorf(\"unable to update job submission: %v\", err)\n\t}\n","sourceCodeStart":1870,"sourceCodeEnd":1906,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/state/state_store.go#L1870-L1906","documentation":"UpsertJob failed while persisting the job into the job_version table via s.upsertJobVersion. Nomad keeps every submitted job version in a separate table for history/diffing; a failure here aborts the registration transaction and the job is not stored.","triggerScenarios":"Registering a job where inserting the current job version into the job_version table fails inside the write transaction (memdb insert error, table corruption, internal bug).","commonSituations":"Seen with long-lived clusters after failed upgrades, or when internal state store invariants break; users observe job register RPC errors.","solutions":["Inspect the wrapped %v cause in server logs","Retry the job submission; the transaction is atomic so no partial versions persist","Verify cluster health and raft snapshot integrity","Upgrade Nomad if the error is reproducible"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Ensure the job parses and has a valid ID/Namespace before submit\nif err := job.Validate(); len(err) > 0 {\n  return fmt.Errorf(\"invalid job: %v\", err)\n}","typeGuard":"func isJobVersionError(err error) bool {\n  return err != nil && strings.Contains(err.Error(), \"job_version table\")\n}","tryCatchPattern":"_, err := client.Jobs().Register(job, nil, nil, nil)\nif isJobVersionError(err) {\n  time.Sleep(time.Second)\n  _, err = client.Jobs().Register(job, nil, nil, nil)\n}\nreturn err","preventionTips":["Retry on transient failures - the txn rolled back fully","Verify raft snapshot integrity after storage incidents","Upgrade all servers to the same Nomad version","Watch server logs for repeated state-store write errors"],"tags":["nomad","state-store","job-version"],"backgroundTag":"job-version-write-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}