hashicorp/nomad · error

CSI.ControllerExpandVolume: %v

Error message

CSI.ControllerExpandVolume: %v

What it means

Returned when the CSI plugin's ControllerExpandVolume gRPC call fails (after retries of timeout/Unavailable/ResourceExhausted). Ignorable errors (e.g. volume deleted out-of-band) are filtered out earlier, so what remains is a genuine expansion failure from the storage backend, wrapped with the 'CSI.ControllerExpandVolume' prefix.

Source

Thrown at client/csi_endpoint.go:266

	csiReq := req.ToCSIRequest()

	ctx, cancelFn := c.requestContext()
	defer cancelFn()

	// CSI ControllerExpandVolume errors for timeout, codes.Unavailable and
	// codes.ResourceExhausted are retried; all other errors are fatal.
	cresp, err := plugin.ControllerExpandVolume(ctx, csiReq,
		grpc_retry.WithPerRetryTimeout(CSIPluginRequestTimeout),
		grpc_retry.WithMax(3),
		grpc_retry.WithBackoff(grpc_retry.BackoffExponential(100*time.Millisecond)))
	if errors.Is(err, nstructs.ErrCSIClientRPCIgnorable) {
		// if the volume was deleted out-of-band, we'll get an error from
		// the plugin but can safely ignore it
		c.c.logger.Debug("could not expand volume", "error", err)
		return nil
	}
	if err != nil {
		return fmt.Errorf("CSI.ControllerExpandVolume: %v", err)
	}
	if cresp == nil {
		c.c.logger.Warn("plugin did not return error or response; this is a bug in the plugin and should be reported to the plugin author")
		return fmt.Errorf("CSI.ControllerExpandVolume: plugin did not return error or response")
	}
	resp.CapacityBytes = cresp.CapacityBytes
	resp.NodeExpansionRequired = cresp.NodeExpansionRequired
	return nil
}

func (c *CSI) ControllerDeleteVolume(req *structs.ClientCSIControllerDeleteVolumeRequest, resp *structs.ClientCSIControllerDeleteVolumeResponse) error {
	defer metrics.MeasureSince([]string{"client", "csi_controller", "delete_volume"}, time.Now())

	plugin, err := c.findControllerPlugin(req.PluginID)
	if err != nil {
		// the server's view of the plugin health is stale, so let it know it
		// should retry with another controller instance
		return fmt.Errorf("CSI.ControllerDeleteVolume: %w: %v",

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Read the wrapped gRPC status to identify the backend reason (NotFound → volume gone; InvalidArgument → check requested capacity/capabilities)
2. Verify the requested new capacity is larger than current and within provider limits
3. Confirm the storage backend and plugin support expansion (some require offline or node-level resize) — check NodeExpansionRequired in the response path
4. Check plugin and provider logs; free capacity/quota if ResourceExhausted after retries
5. Retry after resolving backend-side conditions

// Pre-check: volume exists, new capacity > current, backend supports expansion
vol, err := client.Volumes().Get(volumeID)
if err != nil { return err }
if newCapacityBytes <= vol.CapacityBytes {
    return fmt.Errorf("cannot shrink volume %q from %d to %d", volumeID, vol.CapacityBytes, newCapacityBytes)
}

err := client.ExpandVolume(req)
if err != nil && strings.Contains(err.Error(), "CSI.ControllerExpandVolume:") {
    if s, ok := status.FromError(errors.Unwrap(err)); ok {
        switch s.Code() {
        case codes.NotFound:
            // volume deleted out-of-band: resync state
        case codes.InvalidArgument:
            // fix requested capacity/capabilities
        default:
            // check backend capacity/quota, then retry
        }
    }
}

When it happens

Trigger: plugin.ControllerExpandVolume(ctx, csiReq, ...) returns a non-ignorable err — e.g. backend rejects the resize, requested capacity below current size, filesystem/provider does not support online expansion, gRPC timeout, or fatal codes like InvalidArgument/NotFound.

Common situations: Storage pool full or provider quota limits; attempting to shrink a volume (unsupported); plugin/backend lacking offline-expansion support; volume deleted out-of-band in ways not mapped to the ignorable error; wrong requested capacity units.

Related errors

• nil response from plugin.NodeExpandVolume
• requested capabilities not compatible with volume %q: %v
• controller plugin returned an error: %v
• volume %q cannot be expanded while in use: %v
• expand is not implemented by this controller plugin

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/2a7afeec4501878c. Report an issue: GitHub.