invoke-ai/InvokeAI · error · ValueError

Refusing to record {compute_dtype} as an FP8 compute dtype;

Error message

Refusing to record {compute_dtype} as an FP8 compute dtype; it is a storage-only dtype. This usually means the compute dtype was derived from an already-fp8-cast model.

What it means

set_fp8_compute_dtype() in invokeai/backend/util/fp8.py records, as an attribute on an nn.Module, the dtype that FP8 layerwise-cast layers compute in (fp16/bf16). It refuses the record if the given compute_dtype is one of FP8_STORAGE_DTYPES (torch.float8_e4m3fn / float8_e5m2), because those are storage-only dtypes with no CUDA arithmetic kernels — downstream code trusts this marker when building tensors, so recording float8 would silently reintroduce 'pow_cuda not implemented'-style crashes. The typical cause is deriving the compute dtype from model.dtype of a model that is already fp8-cast (double casting).

Source

Thrown at invokeai/backend/util/fp8.py:38

# Storage-only float8 dtypes. Weights may be held in these, but no math may be done in them.
FP8_STORAGE_DTYPES: tuple[torch.dtype, ...] = (torch.float8_e4m3fn, torch.float8_e5m2)

# Attribute set on a model by the loader when FP8 layerwise casting is applied. It lives in the
# module's `__dict__` (torch.dtype is not a Parameter/Module), so it survives the deepcopy of the
# meta shell in the shared-CPU-weights adoption path.
FP8_COMPUTE_DTYPE_ATTR = "_invokeai_fp8_compute_dtype"


def set_fp8_compute_dtype(model: torch.nn.Module, compute_dtype: torch.dtype) -> None:
    """Record the dtype that `model`'s fp8-cast layers compute in."""
    if compute_dtype in FP8_STORAGE_DTYPES:
        # A float8 compute dtype is never valid, and recording one would silently reintroduce the
        # very crash this module exists to prevent: `get_model_compute_dtype` trusts the marker, so
        # every downstream tensor would be built in a dtype torch has no arithmetic kernels for.
        # The realistic way to get here is deriving the compute dtype from a model that is already
        # cast (i.e. casting twice) — fail loudly at the source instead.
        raise ValueError(
            f"Refusing to record {compute_dtype} as an FP8 compute dtype; it is a storage-only dtype. "
            "This usually means the compute dtype was derived from an already-fp8-cast model."
        )
    setattr(model, FP8_COMPUTE_DTYPE_ATTR, compute_dtype)


def get_model_compute_dtype(model: torch.nn.Module) -> torch.dtype:
    """Return the dtype that `model` actually computes in.

    Equivalent to `model.dtype` for normally-loaded models. For models with FP8 storage it returns
    the compute dtype (fp16/bf16) rather than the float8 storage dtype.
    """
    marked = getattr(model, FP8_COMPUTE_DTYPE_ATTR, None)
    if isinstance(marked, torch.dtype):
        return marked

    dtype = getattr(model, "dtype", None)
    if not isinstance(dtype, torch.dtype):

View on GitHub (pinned to 0b6a024f2f)

Solutions

Use get_model_compute_dtype(model) instead of model.dtype whenever the dtype is needed for compute-side tensors or a re-cast.
If applying FP8 to an already-fp8 model, skip the second cast (check for the FP8_COMPUTE_DTYPE_ATTR marker first).
Pass an explicit fp16/bf16 dtype (e.g. the pipeline's configured precision) to set_fp8_compute_dtype.
Assert with a guard before casting: if getattr(model, FP8_COMPUTE_DTYPE_ATTR, None) is not None: return.

Example fix

// before
compute_dtype = model.dtype  # float8_e4m3fn on an already-cast model
set_fp8_compute_dtype(model, compute_dtype)
// after
from invokeai.backend.util.fp8 import get_model_compute_dtype
compute_dtype = get_model_compute_dtype(model)  # fp16/bf16
set_fp8_compute_dtype(model, compute_dtype)

Defensive patterns

Strategy: validation

Validate before calling

from invokeai.backend.util.fp8 import FP8_STORAGE_DTYPES, get_model_compute_dtype

def safe_fp8_cast(model, compute_dtype=None):
    if compute_dtype is None:
        compute_dtype = get_model_compute_dtype(model)
    assert compute_dtype not in FP8_STORAGE_DTYPES, (
        f'{compute_dtype} is storage-only; use get_model_compute_dtype(model) instead of model.dtype'
    )
    set_fp8_compute_dtype(model, compute_dtype)

Type guard

def is_compute_safe_dtype(dtype: torch.dtype) -> bool:
    return dtype not in (torch.float8_e4m3fn, torch.float8_e5m2)

Try / catch

try:
    set_fp8_compute_dtype(model, compute_dtype)
except ValueError as e:
    if 'storage-only dtype' in str(e):
        compute_dtype = get_model_compute_dtype(model)
        set_fp8_compute_dtype(model, compute_dtype)
    else:
        raise

Prevention

Never read model.dtype on FP8-cast models; always use get_model_compute_dtype(model).
Skip casting entirely if the FP8_COMPUTE_DTYPE_ATTR marker is already present on the module.
Centralize all dtype selection in one helper instead of scattering model.dtype reads.
Add a unit test asserting the chosen compute dtype is not in FP8_STORAGE_DTYPES before any cast.

When it happens

Trigger: Calling set_fp8_compute_dtype(model, model.dtype) or any path (_apply_fp8_to_nn_module / ModelLoader._apply_fp8_layerwise_casting) where the compute dtype was taken from an already-fp8-cast model (model.dtype returns float8_e4m3fn because it is derived from the first parameter), instead of using get_model_compute_dtype(model) or an explicit fp16/bf16 dtype.

Common situations: Custom loader/plugin code reading model.dtype after an initial FP8 cast and passing it into a second cast pass; casting a quantized checkpoint twice; copying dtype from a source model that was loaded with FP8 layerwise casting; building LoRA patch weights from the wrong dtype source.

Related errors

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/fa9bd223d540f592. Report an issue: GitHub.