invoke-ai/InvokeAI · error · ValueError

Cannot force both direct and sidecar patching.

Error message

Cannot force both direct and sidecar patching.

What it means

apply_smart_model_patch() chooses between direct in-place layer patching and sidecar (out-of-place) patching. Forcing both simultaneously is contradictory — direct patching mutates weights in place while sidecar keeps originals intact — so ValueError is raised as an argument-validation guard.

Source

Thrown at invokeai/backend/patches/layer_patcher.py:163

                else:
                    logger = InvokeAILogger.get_logger(LayerPatcher.__name__)
                    logger.warning("Failed to find module for LoRA layer key: %s", layer_key)
                continue

            # Decide whether to use direct patching or a sidecar patch.
            # Direct patching is preferred, because it results in better runtime speed.
            # Reasons to use sidecar patching:
            # - The module is quantized, so the caller passed force_sidecar_patching=True.
            # - The module already has sidecar patches.
            # - The module is on the CPU (and we don't want to store a second full copy of the original weights on the
            #   CPU, since this would double the RAM usage)
            # NOTE: For now, we don't check if the layer is quantized here. We assume that this is checked in the caller
            # and that the caller will set force_sidecar_patching=True if the layer is quantized.
            # TODO(ryand): Handle the case where we are running without a GPU. Should we set a config flag that allows
            # forcing full patching even on the CPU?
            use_sidecar_patching = False
            if force_direct_patching and force_sidecar_patching:
                raise ValueError("Cannot force both direct and sidecar patching.")
            elif force_sidecar_patching:
                use_sidecar_patching = True
            elif LayerPatcher._is_any_part_of_layer_fp8(module):
                # FP8 weights (e.g. a model loaded with fp8_storage layerwise casting) cannot be
                # directly patched: _apply_model_layer_patch does an in-place add on the model weight,
                # and CUDA has no add kernel for float8 ("ufunc_add_CUDA not implemented for
                # Float8_e4m3fn"). Sidecar patching dequantizes to the compute dtype before any math,
                # so it works regardless of the storage dtype. This takes precedence over
                # force_direct_patching, since direct patching is simply not possible on fp8 weights.
                use_sidecar_patching = True
            elif force_direct_patching:
                use_sidecar_patching = False
            elif module.get_num_patches() > 0:
                use_sidecar_patching = True
            elif LayerPatcher._is_any_part_of_layer_on_cpu(module):
                use_sidecar_patching = True

            if use_sidecar_patching:

View on GitHub (pinned to 0b6a024f2f)

Solutions

1. Set only one of force_direct_patching / force_sidecar_patching to True.
2. If the layer is quantized (fp8/bnb), remove force_direct_patching — sidecar is required.
3. If you need direct patching (e.g. CPU without GPU), drop force_sidecar_patching and ensure the layer is not fp8.
4. Audit config flags (e.g. attention-patching options) that feed these booleans.

Example fix

// before
patcher.apply_smart_model_patch(model, 'unet', loras=loras, force_direct_patching=True, force_sidecar_patching=True)
// after
patcher.apply_smart_model_patch(model, 'unet', loras=loras, force_sidecar_patching=True)

if force_direct_patching and force_sidecar_patching:
    raise ValueError('Pick one patching mode: direct (in-place) or sidecar (out-of-place)')
if is_quantized(layer) and force_direct_patching:
    raise ValueError('Quantized layers require sidecar patching')

def patch_mode_is_valid(force_direct: bool, force_sidecar: bool) -> bool:
    return not (force_direct and force_sidecar)

try:
    apply_smart_model_patch(model, prefix, loras, force_direct_patching=fd, force_sidecar_patching=fs)
except ValueError as e:
    if 'Cannot force both' in str(e):
        apply_smart_model_patch(model, prefix, loras, force_sidecar_patching=True)  # safe default
    else:
        raise

When it happens

Trigger: Calling apply_smart_model_patch(..., force_direct_patching=True, force_sidecar_patching=True) — directly or via wrappers like patch_unet whose config flags both modes (e.g. sequential-guidance-style direct patching enabled together with a sidecar-forcing quantized model).

Common situations: Config where both a quantized-model requirement (forces sidecar) and a feature requiring direct patching are enabled at once; custom patching code passing both flags defensively; refactoring that merged two patch call sites.

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/63ef697794ef64b4. Report an issue: GitHub.