hashicorp/nomad · error

volume %q cannot be expanded online: %v

Error message

volume %q cannot be expanded online: %v

What it means

Returned when the CSI plugin's controller replies with gRPC status FailedPrecondition during ControllerExpandVolume. It means the storage backend cannot expand this volume while it is in its current state — typically online expansion of an attached/in-use volume is not supported. The library surfaces this as a distinct message so callers know the volume exists but cannot be grown right now.

Source

Thrown at plugins/csi/client.go:537

		return nil, err
	}
	if err := c.ensureConnected(ctx); err != nil {
		return nil, err
	}

	exReq := req.ToCSIRepresentation()
	resp, err := c.controllerClient.ControllerExpandVolume(ctx, exReq, opts...)
	if err != nil {
		code := status.Code(err)
		switch code {
		case codes.InvalidArgument:
			return nil, fmt.Errorf(
				"requested capabilities not compatible with volume %q: %v",
				req.ExternalVolumeID, err)
		case codes.NotFound:
			err = fmt.Errorf("volume %q could not be found: %v", req.ExternalVolumeID, err)
		case codes.FailedPrecondition:
			err = fmt.Errorf("volume %q cannot be expanded online: %v", req.ExternalVolumeID, err)
		case codes.OutOfRange:
			return nil, fmt.Errorf(
				"unsupported capacity_range for volume %q: %v", req.ExternalVolumeID, err)
		case codes.Internal:
			err = fmt.Errorf("controller plugin returned an internal error, check the plugin allocation logs for more information: %v", err)
		default:
			err = fmt.Errorf("controller plugin returned an error: %v", err)
		}
		return nil, err
	}

	return &ControllerExpandVolumeResponse{
		CapacityBytes:         resp.GetCapacityBytes(),
		NodeExpansionRequired: resp.GetNodeExpansionRequired(),
	}, nil
}

// compareCapabilities returns an error if the 'got' capabilities aren't found

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Detach/unpublish the volume from all nodes, expand it, then re-attach.
2. Stop the allocations using the volume so it can be taken offline during expansion.
3. Check whether the CSI plugin/backend supports online expansion and upgrade the plugin if a newer version adds it.
4. Consider migrating data to a volume type that supports online resize.

Example fix

// before
// expanding while volume is attached to a running allocation
nomad volume status <id>  # shows allocations attached
nomad volume expand <id> -capacity 100Gi
// after
# stop/detach first, then expand
nomad job stop <job-using-volume>
nomad volume expand <id> -capacity 100Gi
nomad job run <job-using-volume>

// only expand volumes with no active claims/allocations
if len(volume.Reads) > 0 || len(volume.Writes) > 0 {
    return fmt.Errorf("volume %s is in use; detach before offline expansion", volume.ID)
}
// check the plugin advertises expand capability
if !pluginSupports(caps, csipbv1.ControllerServiceCapability_RPC_EXPAND_VOLUME) {
    return fmt.Errorf("plugin does not support expansion")
}

func isFailedPreconditionErr(err error) bool {
    if err == nil { return false }
    return strings.Contains(err.Error(), "cannot be expanded online")
}

resp, err := client.ControllerExpandVolume(ctx, req)
if err != nil {
    if strings.Contains(err.Error(), "cannot be expanded online") {
        // schedule: stop allocations -> expand -> restart
        return scheduleOfflineExpansion(req.ExternalVolumeID, req.CapacityRange)
    }
    return err
}

When it happens

Trigger: Calling ControllerExpandVolume on a volume that is currently attached/published and whose plugin or storage backend does not permit online (in-use) expansion, receiving codes.FailedPrecondition.

Common situations: Storage arrays that only support offline resize (volume must be detached first); block-attached volumes on backends requiring unmount before resize; plugin versions lacking ControllerExpandVolume support for published volumes.

Related errors

• nil response from plugin.NodeExpandVolume
• CSI plugin failed to register: %w
• failed to probe plugin: %w
• CSI.ControllerDetachVolume: %v
• CSI.ControllerCreateVolume: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/5d6bcf0fc66d6572. Report an issue: GitHub.