{"record":{"id":"9ded049ff894b10f","repo":"hashicorp/nomad","slug":"csi-controllerattachvolume-w-v-wraps-errcsicl","errorCode":null,"errorMessage":"CSI.ControllerAttachVolume: %w: %v (wraps ErrCSIClientRPCRetryable)","messagePattern":"CSI\\.ControllerAttachVolume: %w: (.+?) \\(wraps ErrCSIClientRPCRetryable\\)","errorType":"exception","errorClass":"ErrCSIClientRPCRetryable","httpStatus":null,"severity":"error","filePath":"client/csi_endpoint.go","lineNumber":96,"sourceCode":"\t}\n\treturn nil\n}\n\n// ControllerAttachVolume is used to attach a volume from a CSI Cluster to\n// the storage node provided in the request.\n//\n// The controller attachment flow currently works as follows:\n// 1. Validate the volume request\n// 2. Call ControllerPublishVolume on the CSI Plugin to trigger a remote attachment\n//\n// In the future this may be expanded to request dynamic secrets for attachment.\nfunc (c *CSI) ControllerAttachVolume(req *structs.ClientCSIControllerAttachVolumeRequest, resp *structs.ClientCSIControllerAttachVolumeResponse) error {\n\tdefer metrics.MeasureSince([]string{\"client\", \"csi_controller\", \"publish_volume\"}, time.Now())\n\tplugin, err := c.findControllerPlugin(req.PluginID)\n\tif err != nil {\n\t\t// the server's view of the plugin health is stale, so let it know it\n\t\t// should retry with another controller instance\n\t\treturn fmt.Errorf(\"CSI.ControllerAttachVolume: %w: %v\",\n\t\t\tnstructs.ErrCSIClientRPCRetryable, err)\n\t}\n\tdefer plugin.Close()\n\n\t// The following block of validation checks should not be reached on a\n\t// real Nomad cluster as all of this data should be validated when registering\n\t// volumes with the cluster. They serve as a defensive check before forwarding\n\t// requests to plugins, and to aid with development.\n\n\tif req.VolumeID == \"\" {\n\t\treturn errors.New(\"CSI.ControllerAttachVolume: VolumeID is required\")\n\t}\n\n\tif req.ClientCSINodeID == \"\" {\n\t\treturn errors.New(\"CSI.ControllerAttachVolume: ClientCSINodeID is required\")\n\t}\n\n\tcsiReq, err := req.ToCSIRequest()","sourceCodeStart":78,"sourceCodeEnd":114,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/csi_endpoint.go#L78-L114","documentation":"ControllerAttachVolume wraps a findControllerPlugin failure in ErrCSIClientRPCRetryable, signaling the server that its view of plugin health is stale and it should retry with another controller instance. The wrapped cause is why no controller plugin could be obtained on this client.","triggerScenarios":"A ClientCSIControllerAttachVolume (publish) RPC targets a client where req.PluginID does not resolve to a healthy controller plugin in the client's plugin registry.","commonSituations":"Controller plugin job stopped, crashed, or rescheduled elsewhere; stale server plugin-health cache; plugin registered as node-only (no controller) so ControllerPublishVolume is unsupported; plugin ID typo after renaming the plugin job.","solutions":["Check `nomad plugin status` and allocation state for the controller plugin job; restart/reschedule it if unhealthy","Retry the publish/unpublish workflow — the sentinel makes it retryable at the server","Ensure the plugin is registered as a controller (not node-only) and req.PluginID matches","Update volume registration to point at the correct plugin ID if the plugin was renamed"],"exampleFix":"// before\nplugin_id = \"efs-plugin\" // job now registers as \"efs-csi-controller\"\n// after\nplugin_id = \"efs-csi-controller\" # match job's plugin stanza","handlingStrategy":"retry","validationCode":"// before publish, confirm the controller plugin is healthy on this client\nif p := clientCSIPlugin(pluginID); p == nil || p.Healthy() != nil {\n    return structs.NewErrRPCCallFailed(clientAddr, \"controller plugin unavailable\")\n}","typeGuard":"func isRetryableCSI(err error) bool {\n    return structs.IsErrRetryable(err)\n}","tryCatchPattern":"err := c.ControllerAttachVolume(req, resp)\nif err != nil {\n    if structs.IsErrRetryable(err) {\n        // retry against another controller instance / schedule another node\n        return structs.NewErrRPCCallFailed(addr, err.Error())\n    }\n    return err\n}","preventionTips":["Run controller plugins on dedicated, monitored nodes","Set restart/service-check policies so crashed plugins recover automatically","Pin plugin_id consistently in job volume blocks and volume registrations","Before publishing, detach stale attachments at the provider"],"tags":["csi","plugin","retryable","attach"],"backgroundTag":"csi-plugin-not-found","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}