{"record":{"id":"516847e4ca298be9","repo":"hashicorp/nomad","slug":"failed-to-retrieve-jobs-w","errorCode":null,"errorMessage":"failed to retrieve jobs: %w","messagePattern":"failed to retrieve jobs: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/operator_endpoint.go","lineNumber":774,"sourceCode":"\top.srv.MeasureRPCRate(\"operator\", structs.RateMetricRead, args)\n\tif authErr != nil {\n\t\treturn structs.ErrPermissionDenied\n\t}\n\n\t// This action requires operator read access.\n\taclObj, err := op.srv.ResolveACL(args)\n\tif err != nil {\n\t\treturn err\n\t} else if !aclObj.AllowOperatorRead() {\n\t\treturn structs.ErrPermissionDenied\n\t}\n\n\tws := memdb.NewWatchSet()\n\n\t// Check for jobs that use Vault but don't have an identity for Vault.\n\tjobsIter, err := op.srv.State().Jobs(ws, state.SortDefault)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to retrieve jobs: %w\", err)\n\t}\n\n\tjobs := []*structs.JobListStub{}\n\tfor raw := jobsIter.Next(); raw != nil; raw = jobsIter.Next() {\n\t\tjob := raw.(*structs.Job)\n\n\tTG_LOOP:\n\t\tfor _, tg := range job.TaskGroups {\n\t\t\tfor _, t := range tg.Tasks {\n\t\t\t\tif t.Vault == nil {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\n\t\t\t\tfoundWID := false\n\t\t\t\tfor _, wid := range t.Identities {\n\t\t\t\t\tif wid.IsVault() {\n\t\t\t\t\t\tfoundWID = true\n\t\t\t\t\t\tbreak","sourceCodeStart":756,"sourceCodeEnd":792,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/operator_endpoint.go#L756-L792","documentation":"UpgradeCheckVaultWorkloadIdentity scans the state store for jobs using Vault without a Vault workload identity. If state.StateStore.Jobs() fails to return the jobs iterator, the error is wrapped with %w and returned to the RPC caller.","triggerScenarios":"op.srv.State().Jobs(ws, state.SortDefault) returns a memdb/state-store error — typically an internal state store failure or invalid store snapshot state during restore/upgrade checks.","commonSituations":"State store corruption or a restore in progress; running the upgrade check immediately after a snapshot restore while the state DB is inconsistent.","solutions":["Retry the operator upgrade check command","Check server logs for the wrapped state store error","Verify state store integrity (latest index consistency, raft state)","Restart the server / restore from a known-good snapshot if corruption is confirmed"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"resp, err := client.Operator().UpgradeCheck(nil)\nif err != nil && strings.Contains(err.Error(), \"failed to retrieve jobs\") {\n    // transient state store failure: back off and retry\n}","preventionTips":["Avoid running upgrade checks mid-restore","Monitor state store health on servers","Retry with backoff before escalating"],"tags":["state-store","vault","workload-identity","upgrade"],"backgroundTag":"state-store-query-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}