invoke-ai/InvokeAI · error · ValueError

Cannot split QKV tensor '{key}': first dimension ({tensor.sh

Error message

Cannot split QKV tensor '{key}': first dimension ({tensor.shape[0]}) is not divisible by 3. The model file may be corrupted or incompatible.

What it means

_convert_z_image_gguf_to_diffusers splits fused attention QKV tensors into separate Q, K, V by dividing the first dimension into three equal parts. If tensor.shape[0] is not divisible by 3 the tensor cannot be a valid fused QKV weight, so the loader raises ValueError rather than producing a silently wrong model. This almost always means the source file is not the expected format rather than a code bug.

Source

Thrown at invokeai/backend/model_manager/load/model_loaders/z_image.py:110

            continue

        # Handle fused QKV weights - need to split
        if ".attention.qkv." in key:
            # Get the layer prefix and suffix
            prefix = key.rsplit(".attention.qkv.", 1)[0]
            suffix = key.rsplit(".attention.qkv.", 1)[1]  # "weight" or "bias"

            # Skip non-weight/bias tensors (e.g., FP8 scale_weight tensors)
            # These are quantization metadata and should not be split
            if suffix not in ("weight", "bias"):
                new_sd[key] = value
                continue

            # Split the fused QKV tensor into Q, K, V
            tensor = value
            if hasattr(tensor, "shape"):
                if tensor.shape[0] % 3 != 0:
                    raise ValueError(
                        f"Cannot split QKV tensor '{key}': first dimension ({tensor.shape[0]}) "
                        "is not divisible by 3. The model file may be corrupted or incompatible."
                    )
                dim = tensor.shape[0] // 3
                q = tensor[:dim]
                k = tensor[dim : 2 * dim]
                v = tensor[2 * dim :]

                new_sd[f"{prefix}.attention.to_q.{suffix}"] = q
                new_sd[f"{prefix}.attention.to_k.{suffix}"] = k
                new_sd[f"{prefix}.attention.to_v.{suffix}"] = v
            continue

        # Handle attention key renaming
        if ".attention." in key:
            new_key = key.replace(".q_norm.", ".norm_q.")
            new_key = new_key.replace(".k_norm.", ".norm_k.")
            new_key = new_key.replace(".attention.out.", ".attention.to_out.0.")

View on GitHub (pinned to 0b6a024f2f)

Solutions

1. Re-download the model file and verify its checksum/size; corruption during download is the most common cause.
2. Confirm the file is actually a Z-Image checkpoint and matches the loader's expected key layout; use the correct loader for other architectures.
3. Open the checkpoint locally and inspect the tensor named in the message to see its true shape and key prefix.
4. Update InvokeAI / the loader in case a new checkpoint variant changed the QKV key-matching heuristic.

import os
expected = 3 * head_count * head_dim
shape0 = tensor.shape[0] if hasattr(tensor, "shape") else None
if shape0 is not None and shape0 % 3 != 0:
    raise ValueError(f"tensor '{key}' has shape0={shape0}, not a valid fused QKV (expected multiple of 3, e.g. {expected})")
if os.path.getsize(model_path) < expected_min_size:
    raise ValueError(f"{model_path} looks truncated; re-download it")

def is_splittable_qkv(tensor, key: str) -> bool:
    return hasattr(tensor, "shape") and "qkv" in key.lower() and tensor.ndim >= 1 and tensor.shape[0] % 3 == 0

try:
    model = loader.load_model(config)
except ValueError as e:
    if "not divisible by 3" in str(e):
        raise RuntimeError(
            f"Z-Image checkpoint is corrupted or incompatible ({e}); re-download and verify the checksum"
        ) from e
    raise

When it happens

Trigger: Loading a Z-Image single-file/GGUF checkpoint via _load_from_singlefile (or the SDNQ path via _load_sdnq_transformer) where a tensor whose key was classified as fused-QKV has first dim % 3 != 0.

Common situations: Truncated or partially downloaded GGUF/single-file checkpoint; wrong architecture file fed to the Z-Image loader (a non-QKV tensor matching the QKV key pattern); checkpoint produced by a different quantization/export tool with differently shaped weights.

Related errors

• Only MistralEncoder_GGUF_Config models are supported here.
• Expected Main_GGUF_ZImage_Config, got {type(config).__name__
• Expected 2D embed_tokens weight tensor, got shape {embed_sha
• Expected ModelPatchRaw for LoRA '{lora.lora.key}', got {type
• not a readable GGUF file: {e}

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/9410abe3d6294f7e. Report an issue: GitHub.