hashicorp/nomad · error

eval broker is enabled; eval broker must be paused to delete

Error message

eval broker is enabled; eval broker must be paused to delete evals

What it means

The Eval.Delete RPC refuses to delete evaluations while the eval broker is enabled. With the broker active, deleting evals directly would corrupt scheduler state (dequeueing evals that no longer exist), so Nomad requires the broker to be paused/disabled first.

Source

Thrown at nomad/eval_endpoint.go:484

	}

	if args.Filter != "" && !e.srv.peersCache.ServersMeetMinimumVersion(
		e.srv.Region(), minVersionEvalDeleteByFilter, true) {
		return fmt.Errorf(
			"all servers must be running version %v or later to delete evals by filter",
			minVersionEvalDeleteByFilter)
	}
	if args.Filter != "" && len(args.EvalIDs) > 0 {
		return fmt.Errorf("evals cannot be deleted by both ID and filter")
	}
	if args.Filter == "" && len(args.EvalIDs) == 0 {
		return fmt.Errorf("evals must be deleted by either ID or filter")
	}

	// The eval broker must be disabled otherwise Nomad's state will likely get
	// wild in a very un-fun way.
	if e.srv.evalBroker.Enabled() {
		return errors.New("eval broker is enabled; eval broker must be paused to delete evals")
	}

	if args.Filter != "" {
		count, index, err := e.deleteEvalsByFilter(args)
		if err != nil {
			return err
		}

		// Update the index and return.
		reply.Index = index
		reply.Count = count
		return nil
	}

	// Grab the state snapshot, so we can look up relevant eval information.
	serverStateSnapshot, err := e.srv.State().Snapshot()
	if err != nil {
		return fmt.Errorf("failed to lookup state snapshot: %v", err)

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Pause the eval broker (agent config / supported enterprise tooling) before deleting evals, then re-enable it
2. Use 'nomad eval purge' only in disaster-recovery scenarios as documented, with the broker disabled
3. Instead of deleting evals, cancel or let them complete; use 'nomad eval list' to triage
4. If doing state surgery, stop/safely pause scheduling, disable the broker, delete, then re-enable

Example fix

// before
nomad eval delete -filter 'Status == "blocked"'
// error: eval broker must be paused
// after: pause broker via operator endpoint, then delete
curl -X POST $NOMAD/v1/operator/scheduler/pause-broker?enabled=true -H "X-Nomad-Token: $TOKEN"
nomad eval delete -filter 'Status == "blocked"'

// shell: only attempt eval deletion when the broker is paused
nomad operator scheduler get-config -json | jq -e '.PauseEvalBroker == true' \
  || { echo 'pause the eval broker first'; exit 1; }

// shell
if ! nomad eval delete "$ID" 2>&1 | grep -q 'eval broker is enabled'; then
  echo "deleted $ID"
else
  echo "pause broker first"
fi

When it happens

Trigger: Calling Eval.Delete (RPC Eval.DeleteEval) with evals still flowing through the broker — i.e. any delete request made on a healthy, non-paused cluster, whether by ID or filter.

Common situations: Operators attempting cleanup of stuck/historical evals via 'nomad eval delete' or the API on an OSS cluster; automation scripts purging evals without pausing the broker first; debugging broker backlogs by mass deletion.

Related errors

• eval not found
• eval broker is enabled; eval broker must be paused to delete
• Job registration, dispatch, and scale are disabled by the sc
• Invalid operator policy: %#v
• Invalid operator capability '%s'

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/67f48ae94ab4e65b. Report an issue: GitHub.