invoke-ai/InvokeAI · error · ValueError

Operation changed the dtype of SDNQTensor unexpectedly.

Error message

Operation changed the dtype of SDNQTensor unexpectedly.

What it means

SDNQTensor (like GGMLTensor) requires operations dispatched through apply_to_quantized_tensor to preserve the quantized_data dtype; a dtype change would break dequantization invariants. Ops such as .to(dtype=torch.float32) on the quantized tensor therefore raise ValueError.

Source

Thrown at invokeai/backend/quantization/sdnq/sdnq_tensor.py:86

    dequantized_args = [process_tensor(a) for a in args]
    dequantized_kwargs = {k: process_tensor(v) for k, v in kwargs.items()}
    return func(*dequantized_args, **dequantized_kwargs)


def apply_to_quantized_tensor(func, args, kwargs):
    """Apply function to quantized tensor and re-wrap result in SDNQTensor.

    Assumes that the first argument is an SDNQTensor.
    """
    sdnq_tensor = args[0]
    assert isinstance(sdnq_tensor, SDNQTensor)
    assert all(not isinstance(a, SDNQTensor) for a in args[1:])
    assert all(not isinstance(v, SDNQTensor) for v in kwargs.values())

    new_data = func(sdnq_tensor.quantized_data, *args[1:], **kwargs)

    if new_data.dtype != sdnq_tensor.quantized_data.dtype:
        raise ValueError("Operation changed the dtype of SDNQTensor unexpectedly.")

    # Realign the auxiliary payloads (scale / zero_point / svd) to the new data's device so the whole
    # wrapper lives on one device. `_to_copy` implements SDNQTensor.to(device); without this a
    # "GPU-resident" SDNQ parameter would keep its scale/zero_point/svd tensors in system RAM,
    # forcing a host->device copy of all of them (both SVD matrices included) on every dequantization
    # of every quantized layer, on every inference step. We only move the device and preserve each
    # tensor's own dtype (a dtype change to the packed data is already rejected above).
    target_device = new_data.device

    def _align(t: Optional[torch.Tensor]) -> Optional[torch.Tensor]:
        if isinstance(t, torch.Tensor) and t.device != target_device:
            return t.to(device=target_device)
        return t

    return SDNQTensor(
        data=new_data,
        quantization_type=sdnq_tensor._quantization_type,
        tensor_shape=sdnq_tensor.tensor_shape,

View on GitHub (pinned to 0b6a024f2f)

Solutions

1. Avoid dtype casts on SDNQTensors; let dequantization apply compute_dtype
2. Use .to(device=...) only when moving devices (payloads keep dtype, aux tensors are realigned)
3. Dequantize to a plain tensor first if a real dtype change is required

Example fix

// before
layer.weight.float()
// after
dequantized = layer.weight.dequantize().float()  # plain tensor, cast is fine

def safe_cast_all(module, dtype):
    for p in module.parameters():
        if isinstance(p, SDNQTensor):
            continue  # quantized params keep their dtype
    module.to(dtype=dtype)  # then fix up if needed via dequantize-aware path

def is_sdnq(t: torch.Tensor) -> bool:
    return isinstance(t, SDNQTensor)

try:
    out = weight.to(dtype=torch.bfloat16)
except ValueError:
    out = weight.dequantize().to(dtype=torch.bfloat16)

When it happens

Trigger: Calling .to(dtype=...), .float(), .half(), or any torch op that changes the dtype of the underlying quantized payload of an SDNQTensor.

Common situations: Generic model-wide casting code (model.half()/model.float()); pipelines that cast all parameters for mixed precision; third-party code assuming every parameter is a plain torch.Tensor.

Related errors

• Operation changed the dtype of GGMLTensor unexpectedly.
• Invalid or expired token
• transformer is SDNQ-quantized; use Main_SDNQ_Diffusers_FLUX_
• transformer is SDNQ-quantized; use Main_SDNQ_Diffusers_Flux2
• transformer is SDNQ-quantized; use Main_SDNQ_Diffusers_ZImag

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/44605680f8ee51a0. Report an issue: GitHub.