invoke-ai/InvokeAI · error · RuntimeError

The base model quantization format (likely bitsandbytes) is

Error message

The base model quantization format (likely bitsandbytes) is not compatible with DoRA patches.

What it means

DoRALayer.get_parameters() needs the original layer weights to compute the DoRA weight decomposition. If any original parameter sits on the 'meta' device, the base weights were never materialized — typical of bitsandbytes quantization — so the required tensors are unavailable and RuntimeError is raised.

Source

Thrown at invokeai/backend/patches/layers/dora_layer.py:99

        out_weight *= self.dora_scale / direction_norm

        return out_weight - orig_weight

    def to(self, device: torch.device | None = None, dtype: torch.dtype | None = None):
        super().to(device=device, dtype=dtype)
        self.up = self.up.to(device=device, dtype=dtype)
        self.down = self.down.to(device=device, dtype=dtype)
        self.dora_scale = self.dora_scale.to(device=device, dtype=dtype)

    def calc_size(self) -> int:
        return super().calc_size() + calc_tensors_size([self.up, self.down, self.dora_scale])

    def get_parameters(self, orig_parameters: dict[str, torch.Tensor], weight: float) -> dict[str, torch.Tensor]:
        if any(p.device.type == "meta" for p in orig_parameters.values()):
            # If any of the original parameters are on the 'meta' device, we assume this is because the base model is in
            # a quantization format that doesn't allow easy dequantization.
            raise RuntimeError(
                "The base model quantization format (likely bitsandbytes) is not compatible with DoRA patches."
            )

        scale = self.scale()
        params = {"weight": self.get_weight(orig_parameters["weight"]) * weight}
        bias = self.get_bias(orig_parameters.get("bias", None))
        if bias is not None:
            params["bias"] = bias * (weight * scale)

        # Reshape all params to match the original module's shape.
        for param_name, param_weight in params.items():
            orig_param = orig_parameters[param_name]
            if param_weight.shape != orig_param.shape:
                params[param_name] = param_weight.reshape(orig_param.shape)

        return params

View on GitHub (pinned to 0b6a024f2f)

Solutions

1. Load the base model unquantized (fp16) so DoRA weights can be computed.
2. Swap the DoRA adapter for a plain LoRA, which supports quantized bases.
3. Check the adapter's config for use_dora: true and convert it to a standard LoRA (dora removal tooling / retraining without DoRA).
4. Verify base-model quantization format in the Model Manager before attaching DoRA patches.

Example fix

// before: bnb-quantized base + DoRA adapter
model = load_model(path, quantization='bnb-nf4'); apply_dora(model, dora_lora)
// after
model = load_model(path, dtype=torch.float16); apply_dora(model, dora_lora)

meta_params = [n for n, p in orig_parameters.items() if p.device.type == 'meta']
if meta_params:
    print(f'Base weights not materialized ({meta_params}); DoRA unsupported — load model unquantized or use plain LoRA')

def dora_compatible(orig_parameters: dict) -> bool:
    return not any(p.device.type == 'meta' for p in orig_parameters.values())

try:
    params = dora_layer.get_parameters(orig_parameters, weight)
except RuntimeError as e:
    if 'not compatible with DoRA' in str(e):
        params = plain_lora_layer.get_parameters(orig_parameters, weight)  # fallback to LoRA
    else:
        raise

When it happens

Trigger: Applying a DoRA LoRA to a model whose linear layers are quantized with bitsandbytes (or another format leaving weights on the meta device), causing `any(p.device.type == 'meta' ...)` to be True inside get_parameters().

Common situations: Loading a 4-bit/8-bit bnb-quantized checkpoint and then applying a DoRA adapter downloaded from the Hub; migrating LoRA configs between a standard model run and a quantized model run without noticing the adapter is DoRA.

Related errors

• state dict does not look like bnb quantized nf4
• filename does not look like bnb quantized llm_int8
• state dict does not look like bnb quantized llm_int8
• Invalid or expired token
• Expected ModelPatchRaw for LoRA '{lora.lora.key}', got {type

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/654b2a4cdc7ec98a. Report an issue: GitHub.