invoke-ai/InvokeAI · error · RuntimeError

Unexpected keys loading Gemma-2 GGUF encoder: {unexpected[:1

Error message

Unexpected keys loading Gemma-2 GGUF encoder: {unexpected[:10]}

What it means

After converting the GGUF state dict and loading it into a meta-initialized Gemma2Model with strict=False, any keys not consumed by the model indicate the converter and the transformers Gemma2Model definition disagree. The loader treats leftover unexpected keys as a hard failure (RuntimeError, first 10 shown) rather than silently dropping weights.

Source

Thrown at invokeai/backend/model_manager/load/model_loaders/gemma2_encoder.py:172

    """
    import accelerate
    from transformers import Gemma2Config, Gemma2Model
    from transformers.modeling_gguf_pytorch_utils import load_gguf_checkpoint

    from invokeai.backend.quantization.gguf.ggml_tensor import GGMLTensor
    from invokeai.backend.quantization.gguf.loaders import gguf_sd_loader

    # Read the Gemma-2 config from the GGUF metadata (avoids re-deriving Gemma-2 defaults), then load the
    # quantized storage as GGMLTensor wrappers.
    gemma_config = Gemma2Config(**load_gguf_checkpoint(str(gguf_path), return_tensors=False)["config"])
    sd = _convert_gemma_llamacpp_to_pytorch(gguf_sd_loader(gguf_path, compute_dtype=compute_dtype))

    with accelerate.init_empty_weights():
        model = Gemma2Model(gemma_config)

    _missing, unexpected = model.load_state_dict(sd, strict=False, assign=True)
    if unexpected:
        raise RuntimeError(f"Unexpected keys loading Gemma-2 GGUF encoder: {unexpected[:10]}")

    # Materialize the weights that cannot remain quantized:
    #  - the token embedding, because nn.Embedding needs indexed access, and
    #  - every RMSNorm weight (1D). Gemma2RMSNorm does `self.weight.float()` and, critically, llama.cpp
    #    folds +1 into the stored norm weight while Gemma2RMSNorm re-adds it at runtime (`1 + weight`), so
    #    we subtract 1 to match transformers' Gemma2TensorProcessor. The large 2D projection weights stay
    #    GGMLTensor and are dequantized on demand by the model cache.
    for module in model.modules():
        for name, param in list(module.named_parameters(recurse=False)):
            if not isinstance(param, GGMLTensor):
                continue
            if isinstance(module, torch.nn.Embedding):
                setattr(module, name, torch.nn.Parameter(param.get_dequantized_tensor(), requires_grad=False))
            elif param.ndim == 1:
                setattr(module, name, torch.nn.Parameter(param.get_dequantized_tensor() - 1.0, requires_grad=False))

    # Re-materialize meta buffers not present in the GGUF (the rotary embedding's inv_freq).
    for name, buf in list(model.named_buffers()):

View on GitHub (pinned to 0b6a024f2f)

Solutions

1. Read the listed unexpected keys and add mappings/skips in _convert_gemma_llamacpp_to_pytorch for them
2. Pin or align the transformers version so Gemma2Model's key names match the converter map
3. Regenerate the GGUF without tensors the encoder cannot consume

Example fix

# before
RuntimeError: Unexpected keys loading Gemma-2 GGUF encoder: ['layers.0.self_attn.kv_proj.weight']
# after
_GEMMA_GGUF_KEY_MAP = {..., "attn_kv": "self_attn.k_proj"}  # key now matches Gemma2Model

def keys_match_model(converted_keys, model):
    model_keys = set(model.state_dict().keys())
    unexpected = [k for k in converted_keys if k not in model_keys]
    return not unexpected, unexpected

def is_valid_state_dict(sd: dict) -> bool:
    model_keys = set(Gemma2Model(gemma_config).state_dict().keys())
    return all(k in model_keys for k in sd)

try:
    model = load_gemma2_model_from_gguf(gguf_path, dtype)
except RuntimeError as e:
    if "Unexpected keys loading Gemma-2 GGUF encoder" in str(e):
        print(f"Converter/model key mismatch: {e}; update the key map or transformers version")
    else:
        raise

When it happens

Trigger: load_gemma2_model_from_gguf on a GGUF whose converted keys don't match Gemma2Model's parameter names — e.g. an extra tensor mapped to a nonexistent module, or a transformers version whose Gemma2 module names changed.

Common situations: transformers upgrade renaming Gemma2 modules (so old mapped names become unexpected); converter map updated without updating the model; GGUF containing tensors the encoder should not load (e.g. output/lm_head) but the converter passes through.

Related errors

• Gemma-2 GGUF encoder has parameters left on the meta device
• Gemma2 GGUF embedding_length {hidden_size} is incompatible w
• state dict does not look like GGUF quantized
• state dict looks like GGUF quantized
• state dict does not look like a T5 encoder (no 'enc.blk.*' k

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/ce46949d6aee3ac4. Report an issue: GitHub.