hashicorp/nomad · error

failed to retrieve jobs: %w

Error message

failed to retrieve jobs: %w

What it means

UpgradeCheckVaultWorkloadIdentity scans the state store for jobs using Vault without a Vault workload identity. If state.StateStore.Jobs() fails to return the jobs iterator, the error is wrapped with %w and returned to the RPC caller.

Source

Thrown at nomad/operator_endpoint.go:774

	op.srv.MeasureRPCRate("operator", structs.RateMetricRead, args)
	if authErr != nil {
		return structs.ErrPermissionDenied
	}

	// This action requires operator read access.
	aclObj, err := op.srv.ResolveACL(args)
	if err != nil {
		return err
	} else if !aclObj.AllowOperatorRead() {
		return structs.ErrPermissionDenied
	}

	ws := memdb.NewWatchSet()

	// Check for jobs that use Vault but don't have an identity for Vault.
	jobsIter, err := op.srv.State().Jobs(ws, state.SortDefault)
	if err != nil {
		return fmt.Errorf("failed to retrieve jobs: %w", err)
	}

	jobs := []*structs.JobListStub{}
	for raw := jobsIter.Next(); raw != nil; raw = jobsIter.Next() {
		job := raw.(*structs.Job)

	TG_LOOP:
		for _, tg := range job.TaskGroups {
			for _, t := range tg.Tasks {
				if t.Vault == nil {
					continue
				}

				foundWID := false
				for _, wid := range t.Identities {
					if wid.IsVault() {
						foundWID = true
						break

View on GitHub (pinned to 482b49bf1a)

Solutions

  1. Retry the operator upgrade check command
  2. Check server logs for the wrapped state store error
  3. Verify state store integrity (latest index consistency, raft state)
  4. Restart the server / restore from a known-good snapshot if corruption is confirmed
Defensive patterns

Strategy: retry

Try / catch

resp, err := client.Operator().UpgradeCheck(nil)
if err != nil && strings.Contains(err.Error(), "failed to retrieve jobs") {
    // transient state store failure: back off and retry
}

Prevention

When it happens

Trigger: op.srv.State().Jobs(ws, state.SortDefault) returns a memdb/state-store error — typically an internal state store failure or invalid store snapshot state during restore/upgrade checks.

Common situations: State store corruption or a restore in progress; running the upgrade check immediately after a snapshot restore while the state DB is inconsistent.

Related errors


AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/516847e4ca298be9. Report an issue: GitHub.