{"record":{"id":"2a7afeec4501878c","repo":"hashicorp/nomad","slug":"csi-controllerexpandvolume-v","errorCode":null,"errorMessage":"CSI.ControllerExpandVolume: %v","messagePattern":"CSI\\.ControllerExpandVolume: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/csi_endpoint.go","lineNumber":266,"sourceCode":"\tcsiReq := req.ToCSIRequest()\n\n\tctx, cancelFn := c.requestContext()\n\tdefer cancelFn()\n\n\t// CSI ControllerExpandVolume errors for timeout, codes.Unavailable and\n\t// codes.ResourceExhausted are retried; all other errors are fatal.\n\tcresp, err := plugin.ControllerExpandVolume(ctx, csiReq,\n\t\tgrpc_retry.WithPerRetryTimeout(CSIPluginRequestTimeout),\n\t\tgrpc_retry.WithMax(3),\n\t\tgrpc_retry.WithBackoff(grpc_retry.BackoffExponential(100*time.Millisecond)))\n\tif errors.Is(err, nstructs.ErrCSIClientRPCIgnorable) {\n\t\t// if the volume was deleted out-of-band, we'll get an error from\n\t\t// the plugin but can safely ignore it\n\t\tc.c.logger.Debug(\"could not expand volume\", \"error\", err)\n\t\treturn nil\n\t}\n\tif err != nil {\n\t\treturn fmt.Errorf(\"CSI.ControllerExpandVolume: %v\", err)\n\t}\n\tif cresp == nil {\n\t\tc.c.logger.Warn(\"plugin did not return error or response; this is a bug in the plugin and should be reported to the plugin author\")\n\t\treturn fmt.Errorf(\"CSI.ControllerExpandVolume: plugin did not return error or response\")\n\t}\n\tresp.CapacityBytes = cresp.CapacityBytes\n\tresp.NodeExpansionRequired = cresp.NodeExpansionRequired\n\treturn nil\n}\n\nfunc (c *CSI) ControllerDeleteVolume(req *structs.ClientCSIControllerDeleteVolumeRequest, resp *structs.ClientCSIControllerDeleteVolumeResponse) error {\n\tdefer metrics.MeasureSince([]string{\"client\", \"csi_controller\", \"delete_volume\"}, time.Now())\n\n\tplugin, err := c.findControllerPlugin(req.PluginID)\n\tif err != nil {\n\t\t// the server's view of the plugin health is stale, so let it know it\n\t\t// should retry with another controller instance\n\t\treturn fmt.Errorf(\"CSI.ControllerDeleteVolume: %w: %v\",","sourceCodeStart":248,"sourceCodeEnd":284,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/csi_endpoint.go#L248-L284","documentation":"Returned when the CSI plugin's ControllerExpandVolume gRPC call fails (after retries of timeout/Unavailable/ResourceExhausted). Ignorable errors (e.g. volume deleted out-of-band) are filtered out earlier, so what remains is a genuine expansion failure from the storage backend, wrapped with the 'CSI.ControllerExpandVolume' prefix.","triggerScenarios":"plugin.ControllerExpandVolume(ctx, csiReq, ...) returns a non-ignorable err — e.g. backend rejects the resize, requested capacity below current size, filesystem/provider does not support online expansion, gRPC timeout, or fatal codes like InvalidArgument/NotFound.","commonSituations":"Storage pool full or provider quota limits; attempting to shrink a volume (unsupported); plugin/backend lacking offline-expansion support; volume deleted out-of-band in ways not mapped to the ignorable error; wrong requested capacity units.","solutions":["Read the wrapped gRPC status to identify the backend reason (NotFound → volume gone; InvalidArgument → check requested capacity/capabilities)","Verify the requested new capacity is larger than current and within provider limits","Confirm the storage backend and plugin support expansion (some require offline or node-level resize) — check NodeExpansionRequired in the response path","Check plugin and provider logs; free capacity/quota if ResourceExhausted after retries","Retry after resolving backend-side conditions"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// Pre-check: volume exists, new capacity > current, backend supports expansion\nvol, err := client.Volumes().Get(volumeID)\nif err != nil { return err }\nif newCapacityBytes <= vol.CapacityBytes {\n    return fmt.Errorf(\"cannot shrink volume %q from %d to %d\", volumeID, vol.CapacityBytes, newCapacityBytes)\n}","typeGuard":null,"tryCatchPattern":"err := client.ExpandVolume(req)\nif err != nil && strings.Contains(err.Error(), \"CSI.ControllerExpandVolume:\") {\n    if s, ok := status.FromError(errors.Unwrap(err)); ok {\n        switch s.Code() {\n        case codes.NotFound:\n            // volume deleted out-of-band: resync state\n        case codes.InvalidArgument:\n            // fix requested capacity/capabilities\n        default:\n            // check backend capacity/quota, then retry\n        }\n    }\n}","preventionTips":["Only request capacity increases, never decreases","Confirm the storage backend and plugin support (online) expansion before relying on it","Monitor storage pool utilization to avoid ResourceExhausted during resize","Handle the out-of-band deletion case by reconciling volume state after NotFound errors"],"tags":["csi","grpc","volume-expansion","storage-backend"],"backgroundTag":"grpc-unavailable","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}