{"record":{"id":"f06898e29efc7a80","repo":"hashicorp/nomad","slug":"csi-controllerlistsnapshots-w-v","errorCode":null,"errorMessage":"CSI.ControllerListSnapshots: %w: %v","messagePattern":"CSI\\.ControllerListSnapshots: %w: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"client/csi_endpoint.go","lineNumber":454,"sourceCode":"\t\t// if the snapshot was deleted out-of-band, we'll get an error from\n\t\t// the plugin but can safely ignore it\n\t\tc.c.logger.Debug(\"could not delete snapshot\", \"error\", err)\n\t\treturn nil\n\t}\n\tif err != nil {\n\t\treturn fmt.Errorf(\"CSI.ControllerDeleteSnapshot: %v\", err)\n\t}\n\treturn err\n}\n\nfunc (c *CSI) ControllerListSnapshots(req *structs.ClientCSIControllerListSnapshotsRequest, resp *structs.ClientCSIControllerListSnapshotsResponse) error {\n\tdefer metrics.MeasureSince([]string{\"client\", \"csi_controller\", \"list_snapshots\"}, time.Now())\n\n\tplugin, err := c.findControllerPlugin(req.PluginID)\n\tif err != nil {\n\t\t// the server's view of the plugin health is stale, so let it know it\n\t\t// should retry with another controller instance\n\t\treturn fmt.Errorf(\"CSI.ControllerListSnapshots: %w: %v\",\n\t\t\tnstructs.ErrCSIClientRPCRetryable, err)\n\t}\n\tdefer plugin.Close()\n\n\tcsiReq := req.ToCSIRequest()\n\n\tctx, cancelFn := c.requestContext()\n\tdefer cancelFn()\n\n\t// CSI ControllerListSnapshots errors for timeout, codes.Unavailable and\n\t// codes.ResourceExhausted are retried; all other errors are fatal.\n\tcresp, err := plugin.ControllerListSnapshots(ctx, csiReq,\n\t\tgrpc_retry.WithPerRetryTimeout(CSIPluginRequestTimeout),\n\t\tgrpc_retry.WithMax(3),\n\t\tgrpc_retry.WithBackoff(grpc_retry.BackoffExponential(100*time.Millisecond)))\n\tif err != nil {\n\t\treturn fmt.Errorf(\"CSI.ControllerListSnapshots: %v\", err)\n\t}","sourceCodeStart":436,"sourceCodeEnd":472,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/csi_endpoint.go#L436-L472","documentation":"CSI.ControllerListSnapshots wraps the server RPC ControllerListSnapshots. As with other controller RPCs, a failure from findControllerPlugin (no healthy controller plugin for req.PluginID on this client) is wrapped with structs.ErrCSIClientRPCRetryable so the server knows to retry with another controller instance.","triggerScenarios":"ListSnapshots arrives at a client lacking a registered healthy controller for req.PluginID — plugin crashed, only node plugin running, or registration is stale.","commonSituations":"Listing snapshots while the controller plugin is restarting during an update; multi-client fleets where only some clients run the controller; plugin ID mismatches after job rename.","solutions":["Retry; the RPC will be routed to another healthy controller","Confirm a healthy controller plugin via `nomad plugin status` and the plugin job status","Correct the plugin_id used for snapshot listing queries","Ensure the controller plugin serves the CSI Controller service and is not stuck in pending"],"exampleFix":"# before\n$ nomad snapshot list -plugin broken-id\n# after\n$ nomad plugin status   # find the healthy plugin id first\n$ nomad snapshot list -plugin healthy-controller-id","handlingStrategy":"retry","validationCode":"// Verify controller health before listing:\n// nomad plugin status\n// Restart/reroute if the target controller plugin shows unhealthy.","typeGuard":null,"tryCatchPattern":"// Retryable marker: server will retry on another controller\nif err := c.ControllerListSnapshots(req, resp); err != nil {\n    if errors.Is(err, structs.ErrCSIClientRPCRetryable) {\n        // retry with backoff\n    }\n}","preventionTips":["Run controllers on multiple clients for availability","Use stable plugin IDs across plugin job updates","Confirm registration completes before listing","Alert on `nomad plugin status` deregistrations"],"tags":["csi","snapshot","plugin","retry"],"backgroundTag":"csi-plugin-not-found-retryable","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}