hashicorp/nomad · error

unable to find eval id %q

Error message

unable to find eval id %q

What it means

Returned by updateEvalModifyIndex when the evaluation ID supplied by plan apply does not exist in the "evals" table — the txn.First lookup succeeded but returned nil. The eval ID is logged at error level along with this wrapped error. It means plan apply referenced an eval that was deleted or never inserted.

Source

Thrown at nomad/state/state_store.go:3576

	}
	if err := txn.Insert("index", &IndexEntry{"evals", index}); err != nil {
		return fmt.Errorf("index update failed: %v", err)
	}
	return nil
}

// updateEvalModifyIndex is used to update the modify index of an evaluation that has been
// through a scheduler pass. This is done as part of plan apply. It ensures that when a subsequent
// scheduler workers process a re-queued evaluation it sees any partial updates from the plan apply.
func (s *StateStore) updateEvalModifyIndex(txn *txn, index uint64, evalID string) error {
	// Lookup the evaluation
	existing, err := txn.First("evals", "id", evalID)
	if err != nil {
		return fmt.Errorf("eval lookup failed: %v", err)
	}
	if existing == nil {
		s.logger.Error("unable to find eval", "eval_id", evalID)
		return fmt.Errorf("unable to find eval id %q", evalID)
	}
	eval := existing.(*structs.Evaluation).Copy()
	// Update the indexes
	eval.ModifyIndex = index

	// Insert the eval
	if err := txn.Insert("evals", eval); err != nil {
		return fmt.Errorf("eval insert failed: %v", err)
	}
	if err := txn.Insert("index", &IndexEntry{"evals", index}); err != nil {
		return fmt.Errorf("index update failed: %v", err)
	}
	return nil
}

// DeleteEvalsByFilter is used to delete all evals that are both safe to delete
// and match a filter.
func (s *StateStore) DeleteEvalsByFilter(index uint64, filterExpr string, pageToken string, perPage int32) error {

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Verify the eval still exists via `nomad eval status <eval-id>` / the evaluations API before plan apply
2. Re-run the scheduler; a fresh eval will be created and the stale apply discarded
3. Reduce eval GC aggressiveness (eval_gc_threshold) if evals are being reaped too early
4. Ensure all servers run the same Nomad version during upgrades

Example fix

// before: applying plan with stale eval id
err := s.updateEvalModifyIndex(txn, idx, evalID)
// after
existing, _ := txn.First("evals", "id", evalID)
if existing != nil {
    err = s.updateEvalModifyIndex(txn, idx, evalID)
}

// Go: confirm the eval exists before plan apply
existing, err := txn.First("evals", "id", evalID)
if err != nil {
    return err
}
if existing == nil {
    return fmt.Errorf("eval %q no longer exists; skipping modify-index update", evalID)
}

func evalExists(txn *txn, evalID string) bool {
    raw, err := txn.First("evals", "id", evalID)
    return err == nil && raw != nil
}

// Go
if err != nil && strings.Contains(err.Error(), "unable to find eval id") {
    // stale plan result; drop it and let the scheduler re-queue
    return nil
}

When it happens

Trigger: updateEvalModifyIndex called with an evalID whose evaluation was garbage-collected (DeleteEvalsByFilter), cancelled and purged, or whose ID was mistyped/never created before plan apply ran.

Common situations: Stale scheduler workers applying plans after the eval was deleted; eval GC racing with plan apply; manually purged evals via API; version-mismatch between leader and followers mid-upgrade.

Related errors

• eval broker is enabled; eval broker must be paused to delete
• eval not found
• node not found: %s
• node not found
• error parsing: root should be an object

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/cfde1b211d34e45b. Report an issue: GitHub.