{"record":{"id":"3a61e550055e2412","repo":"hashicorp/nomad","slug":"getting-job-scaling-policies-for-deletion-failed","errorCode":null,"errorMessage":"getting job scaling policies for deletion failed: %v","messagePattern":"getting job scaling policies for deletion failed: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/state/state_store.go","lineNumber":2133,"sourceCode":"\t}\n\n\t// Delete task group volume claims\n\tif err = s.deleteTaskGroupHostVolumeClaimByNamespaceAndJob(index, txn, namespace, jobID); err != nil {\n\t\treturn fmt.Errorf(\"deleting job volume claims failed: %v\", err)\n\t}\n\n\tif err := txn.Insert(\"index\", &IndexEntry{\"scaling_event\", index}); err != nil {\n\t\treturn fmt.Errorf(\"index update failed: %v\", err)\n\t}\n\n\treturn nil\n}\n\n// deleteJobScalingPolicies deletes any scaling policies associated with the job\nfunc (s *StateStore) deleteJobScalingPolicies(index uint64, job *structs.Job, txn *txn) error {\n\titer, err := s.ScalingPoliciesByJobTxn(nil, job.Namespace, job.ID, txn)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"getting job scaling policies for deletion failed: %v\", err)\n\t}\n\n\t// Put them into a slice so there are no safety concerns while actually\n\t// performing the deletes\n\tpolicies := []any{}\n\tfor {\n\t\traw := iter.Next()\n\t\tif raw == nil {\n\t\t\tbreak\n\t\t}\n\t\tpolicies = append(policies, raw)\n\t}\n\n\t// Do the deletes\n\tfor _, p := range policies {\n\t\tif err := txn.Delete(\"scaling_policy\", p); err != nil {\n\t\t\treturn fmt.Errorf(\"deleting scaling policy failed: %v\", err)\n\t\t}","sourceCodeStart":2115,"sourceCodeEnd":2151,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/state/state_store.go#L2115-L2151","documentation":"deleteJobScalingPolicies first lists the job's scaling policies via ScalingPoliciesByJobTxn before deleting them. This wrapped error means that read/iteration failed, so the policy cleanup (and thus job deletion) aborts. It is a wrapped iterator/lookup error, not a delete error.","triggerScenarios":"Job deregistration when ScalingPoliciesByJobTxn(nil, namespace, jobID, txn) errors — e.g. the underlying memdb iterator fails because the transaction is already aborted, or the \"scaling_policy\" table/index is unavailable or corrupted.","commonSituations":"Seen when an earlier step of the deleteJob transaction already failed (aborted txn reused), during snapshot/restore windows, or in tests driving the StateStore with malformed transactions.","solutions":["Inspect the wrapped inner error to identify the failing memdb read.","Retry the job deletion with a fresh write transaction.","Ensure earlier statements in the same txn did not abort it before this call.","Check state store integrity (scaling_policy table) if the error persists across retries."],"exampleFix":"// before\niter, err := s.ScalingPoliciesByJobTxn(nil, job.Namespace, job.ID, txn)\nif err != nil {\n\treturn fmt.Errorf(\"getting job scaling policies for deletion failed: %v\", err)\n}\n// after (use %w so callers can errors.Is on the memdb cause)\nif err != nil {\n\treturn fmt.Errorf(\"getting job scaling policies for deletion failed: %w\", err)\n}","handlingStrategy":"retry","validationCode":"// verify the job exists before policy cleanup\njob, err := state.JobByID(nil, namespace, jobID)\nif err != nil || job == nil {\n\treturn fmt.Errorf(\"job %s/%s not found or read failed: %w\", namespace, jobID, err)\n}","typeGuard":null,"tryCatchPattern":"err := state.DeleteJob(index, ns, jobID)\nif err != nil && strings.Contains(err.Error(), \"getting job scaling policies\") {\n\ttime.Sleep(backoff)\n\treturn state.DeleteJob(index, ns, jobID)\n}","preventionTips":["Never continue using a transaction after an earlier error — abort it and start a new one.","Retry job deletion; policy listing is re-executed on a fresh txn.","Check scaling_policy table consistency after snapshot restores.","Monitor memdb health for read-path errors under load."],"tags":["nomad","state-store","scaling-policies","job-deletion"],"backgroundTag":"state-store-read-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}