hashicorp/nomad · error

DeleteScalingPolicies of removed policies failed: %v

Error message

DeleteScalingPolicies of removed policies failed: %v

What it means

Wraps an error from DeleteScalingPoliciesTxn when removing scaling policies whose targets no longer exist in the updated job. Deletion happens inside the same raft transaction as the job update, so any failure aborts the whole registration.

Source

Thrown at nomad/state/state_store.go:5774

	newTargets := map[string]bool{}
	for _, p := range scalingPolicies {
		newTargets[p.JobKey()] = true
	}
	// find existing policies that need to be deleted
	deletedPolicies := []string{}
	iter, err := s.ScalingPoliciesByJobTxn(ws, job.Namespace, job.ID, txn)
	if err != nil {
		return fmt.Errorf("ScalingPoliciesByJob lookup failed: %v", err)
	}
	for raw := iter.Next(); raw != nil; raw = iter.Next() {
		oldPolicy := raw.(*structs.ScalingPolicy)
		if !newTargets[oldPolicy.JobKey()] {
			deletedPolicies = append(deletedPolicies, oldPolicy.ID)
		}
	}
	err = s.DeleteScalingPoliciesTxn(index, deletedPolicies, txn)
	if err != nil {
		return fmt.Errorf("DeleteScalingPolicies of removed policies failed: %v", err)
	}

	err = s.UpsertScalingPoliciesTxn(index, scalingPolicies, txn)
	if err != nil {
		return fmt.Errorf("UpsertScalingPolicies of policies failed: %v", err)
	}

	return nil
}

// updateJobSubmission stores the original job source and variables associated that the
// job structure originates from. It is up to the job submitter to include the source
// material, and as such sub may be nil, in which case nothing is stored.
func (s *StateStore) updateJobSubmission(index uint64, sub *structs.JobSubmission, namespace, jobID string, version uint64, txn *txn) error {
	// critical that we operate on a copy; the original must not be modified
	// e.g. in the case of job gc and its last second version bump
	sub = sub.Copy()

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Inspect the wrapped inner error
2. Resubmit the job; nothing was persisted so retry is safe
3. If persistent, snapshot/restore the server state

if err != nil && strings.Contains(err.Error(), "DeleteScalingPolicies of removed policies failed") {
    // txn aborted atomically; resubmit the job plan
    return retryJobSubmission(job)
}

When it happens

Trigger: UpsertJob where deletedPolicies is non-empty (job removed a scaling policy block for a task/group) and DeleteScalingPoliciesTxn fails, e.g. memdb delete error or missing row.

Common situations: Editing a job to remove scaling policy blocks; failure indicates state store write issues rather than user error.

Related errors

• unable to update job scaling policies: %v
• getting job scaling policies for deletion failed: %v
• deleting scaling policy failed: %v
• ScalingPoliciesByJob lookup failed: %v
• UpsertScalingPolicies of policies failed: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/d202ee9e16586ed7. Report an issue: GitHub.