{"record":{"id":"25069a33363b241f","repo":"hashicorp/nomad","slug":"failed-to-retrieve-nodes-w","errorCode":null,"errorMessage":"failed to retrieve nodes: %w","messagePattern":"failed to retrieve nodes: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/operator_endpoint.go","lineNumber":807,"sourceCode":"\t\t\t\tfor _, wid := range t.Identities {\n\t\t\t\t\tif wid.IsVault() {\n\t\t\t\t\t\tfoundWID = true\n\t\t\t\t\t\tbreak\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tif !foundWID {\n\t\t\t\t\tjobs = append(jobs, job.Stub(nil, nil))\n\t\t\t\t\tbreak TG_LOOP\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\treply.JobsWithoutVaultIdentity = jobs\n\n\t// Find nodes that don't support workload identities for Vault.\n\tnodesIter, err := op.srv.State().Nodes(ws)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to retrieve nodes: %w\", err)\n\t}\n\n\tnodes := []*structs.NodeListStub{}\n\tfor raw := nodesIter.Next(); raw != nil; raw = nodesIter.Next() {\n\t\tnode := raw.(*structs.Node)\n\n\t\tv, err := version.NewVersion(node.Attributes[\"nomad.version\"])\n\t\tif err != nil || v.LessThan(structs.MinNomadVersionVaultWID) {\n\t\t\tnodes = append(nodes, node.Stub(nil))\n\t\t\tcontinue\n\t\t}\n\t}\n\treply.OutdatedNodes = nodes\n\n\treply.QueryMeta.Index, _ = op.srv.State().LatestIndex()\n\top.srv.setQueryMeta(&reply.QueryMeta)\n\n\treturn nil","sourceCodeStart":789,"sourceCodeEnd":825,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/operator_endpoint.go#L789-L825","documentation":"Same upgrade check as the jobs pass, but for nodes: if state.StateStore.Nodes() fails to iterate nodes, the error is wrapped and returned. Indicates a failure reading the node table from the in-memory state store.","triggerScenarios":"op.srv.State().Nodes(ws) returns an error from the memdb-backed state store — internal query failure or store inconsistency (e.g. mid-restore).","commonSituations":"Running the Vault WI upgrade check against a server with a corrupted or mid-restore state store.","solutions":["Retry the upgrade check","Inspect server logs for the wrapped underlying error","Validate raft/state store health; restore from a good snapshot if corrupt","Ensure the operation is hitting the current leader"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"resp, err := client.Operator().UpgradeCheck(nil)\nif err != nil && strings.Contains(err.Error(), \"failed to retrieve nodes\") {\n    // state store read failure: retry after checking server health\n}","preventionTips":["Run the check against a healthy current leader","Avoid running during snapshot restore","Retry with backoff"],"tags":["state-store","nodes","vault","upgrade"],"backgroundTag":"state-store-query-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}