{"record":{"id":"44605680f8ee51a0","repo":"invoke-ai/InvokeAI","slug":"operation-changed-the-dtype-of-sdnqtensor-unexpect","errorCode":null,"errorMessage":"Operation changed the dtype of SDNQTensor unexpectedly.","messagePattern":"Operation changed the dtype of SDNQTensor unexpectedly\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"invokeai/backend/quantization/sdnq/sdnq_tensor.py","lineNumber":86,"sourceCode":"    dequantized_args = [process_tensor(a) for a in args]\n    dequantized_kwargs = {k: process_tensor(v) for k, v in kwargs.items()}\n    return func(*dequantized_args, **dequantized_kwargs)\n\n\ndef apply_to_quantized_tensor(func, args, kwargs):\n    \"\"\"Apply function to quantized tensor and re-wrap result in SDNQTensor.\n\n    Assumes that the first argument is an SDNQTensor.\n    \"\"\"\n    sdnq_tensor = args[0]\n    assert isinstance(sdnq_tensor, SDNQTensor)\n    assert all(not isinstance(a, SDNQTensor) for a in args[1:])\n    assert all(not isinstance(v, SDNQTensor) for v in kwargs.values())\n\n    new_data = func(sdnq_tensor.quantized_data, *args[1:], **kwargs)\n\n    if new_data.dtype != sdnq_tensor.quantized_data.dtype:\n        raise ValueError(\"Operation changed the dtype of SDNQTensor unexpectedly.\")\n\n    # Realign the auxiliary payloads (scale / zero_point / svd) to the new data's device so the whole\n    # wrapper lives on one device. `_to_copy` implements SDNQTensor.to(device); without this a\n    # \"GPU-resident\" SDNQ parameter would keep its scale/zero_point/svd tensors in system RAM,\n    # forcing a host->device copy of all of them (both SVD matrices included) on every dequantization\n    # of every quantized layer, on every inference step. We only move the device and preserve each\n    # tensor's own dtype (a dtype change to the packed data is already rejected above).\n    target_device = new_data.device\n\n    def _align(t: Optional[torch.Tensor]) -> Optional[torch.Tensor]:\n        if isinstance(t, torch.Tensor) and t.device != target_device:\n            return t.to(device=target_device)\n        return t\n\n    return SDNQTensor(\n        data=new_data,\n        quantization_type=sdnq_tensor._quantization_type,\n        tensor_shape=sdnq_tensor.tensor_shape,","sourceCodeStart":68,"sourceCodeEnd":104,"githubUrl":"https://github.com/invoke-ai/InvokeAI/blob/0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06/invokeai/backend/quantization/sdnq/sdnq_tensor.py#L68-L104","documentation":"SDNQTensor (like GGMLTensor) requires operations dispatched through apply_to_quantized_tensor to preserve the quantized_data dtype; a dtype change would break dequantization invariants. Ops such as .to(dtype=torch.float32) on the quantized tensor therefore raise ValueError.","triggerScenarios":"Calling .to(dtype=...), .float(), .half(), or any torch op that changes the dtype of the underlying quantized payload of an SDNQTensor.","commonSituations":"Generic model-wide casting code (model.half()/model.float()); pipelines that cast all parameters for mixed precision; third-party code assuming every parameter is a plain torch.Tensor.","solutions":["Avoid dtype casts on SDNQTensors; let dequantization apply compute_dtype","Use .to(device=...) only when moving devices (payloads keep dtype, aux tensors are realigned)","Dequantize to a plain tensor first if a real dtype change is required"],"exampleFix":"// before\nlayer.weight.float()\n// after\ndequantized = layer.weight.dequantize().float()  # plain tensor, cast is fine","handlingStrategy":"type-guard","validationCode":"def safe_cast_all(module, dtype):\n    for p in module.parameters():\n        if isinstance(p, SDNQTensor):\n            continue  # quantized params keep their dtype\n    module.to(dtype=dtype)  # then fix up if needed via dequantize-aware path","typeGuard":"def is_sdnq(t: torch.Tensor) -> bool:\n    return isinstance(t, SDNQTensor)","tryCatchPattern":"try:\n    out = weight.to(dtype=torch.bfloat16)\nexcept ValueError:\n    out = weight.dequantize().to(dtype=torch.bfloat16)","preventionTips":["Exclude SDNQTensor parameters from precision-casting utilities","Move devices with .to(device=...) only; dtype changes go through dequantize","Guard generic autocast/casting helpers with isinstance checks on quantized tensor types"],"tags":["valueerror","quantization","sdnq","dtype"],"backgroundTag":"quantized-dtype-change","analyzedSha":"0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06","analyzedAt":"2026-08-29T04:46:49.967Z","schemaVersion":2},"datasetVersion":"2026-08-29T07:17:48.351Z"}