{"record":{"id":"ce46949d6aee3ac4","repo":"invoke-ai/InvokeAI","slug":"unexpected-keys-loading-gemma-2-gguf-encoder-une","errorCode":null,"errorMessage":"Unexpected keys loading Gemma-2 GGUF encoder: {unexpected[:10]}","messagePattern":"Unexpected keys loading Gemma-2 GGUF encoder: (.+?)","errorType":"validation","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"invokeai/backend/model_manager/load/model_loaders/gemma2_encoder.py","lineNumber":172,"sourceCode":"    \"\"\"\n    import accelerate\n    from transformers import Gemma2Config, Gemma2Model\n    from transformers.modeling_gguf_pytorch_utils import load_gguf_checkpoint\n\n    from invokeai.backend.quantization.gguf.ggml_tensor import GGMLTensor\n    from invokeai.backend.quantization.gguf.loaders import gguf_sd_loader\n\n    # Read the Gemma-2 config from the GGUF metadata (avoids re-deriving Gemma-2 defaults), then load the\n    # quantized storage as GGMLTensor wrappers.\n    gemma_config = Gemma2Config(**load_gguf_checkpoint(str(gguf_path), return_tensors=False)[\"config\"])\n    sd = _convert_gemma_llamacpp_to_pytorch(gguf_sd_loader(gguf_path, compute_dtype=compute_dtype))\n\n    with accelerate.init_empty_weights():\n        model = Gemma2Model(gemma_config)\n\n    _missing, unexpected = model.load_state_dict(sd, strict=False, assign=True)\n    if unexpected:\n        raise RuntimeError(f\"Unexpected keys loading Gemma-2 GGUF encoder: {unexpected[:10]}\")\n\n    # Materialize the weights that cannot remain quantized:\n    #  - the token embedding, because nn.Embedding needs indexed access, and\n    #  - every RMSNorm weight (1D). Gemma2RMSNorm does `self.weight.float()` and, critically, llama.cpp\n    #    folds +1 into the stored norm weight while Gemma2RMSNorm re-adds it at runtime (`1 + weight`), so\n    #    we subtract 1 to match transformers' Gemma2TensorProcessor. The large 2D projection weights stay\n    #    GGMLTensor and are dequantized on demand by the model cache.\n    for module in model.modules():\n        for name, param in list(module.named_parameters(recurse=False)):\n            if not isinstance(param, GGMLTensor):\n                continue\n            if isinstance(module, torch.nn.Embedding):\n                setattr(module, name, torch.nn.Parameter(param.get_dequantized_tensor(), requires_grad=False))\n            elif param.ndim == 1:\n                setattr(module, name, torch.nn.Parameter(param.get_dequantized_tensor() - 1.0, requires_grad=False))\n\n    # Re-materialize meta buffers not present in the GGUF (the rotary embedding's inv_freq).\n    for name, buf in list(model.named_buffers()):","sourceCodeStart":154,"sourceCodeEnd":190,"githubUrl":"https://github.com/invoke-ai/InvokeAI/blob/0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06/invokeai/backend/model_manager/load/model_loaders/gemma2_encoder.py#L154-L190","documentation":"After converting the GGUF state dict and loading it into a meta-initialized Gemma2Model with strict=False, any keys not consumed by the model indicate the converter and the transformers Gemma2Model definition disagree. The loader treats leftover unexpected keys as a hard failure (RuntimeError, first 10 shown) rather than silently dropping weights.","triggerScenarios":"load_gemma2_model_from_gguf on a GGUF whose converted keys don't match Gemma2Model's parameter names — e.g. an extra tensor mapped to a nonexistent module, or a transformers version whose Gemma2 module names changed.","commonSituations":"transformers upgrade renaming Gemma2 modules (so old mapped names become unexpected); converter map updated without updating the model; GGUF containing tensors the encoder should not load (e.g. output/lm_head) but the converter passes through.","solutions":["Read the listed unexpected keys and add mappings/skips in _convert_gemma_llamacpp_to_pytorch for them","Pin or align the transformers version so Gemma2Model's key names match the converter map","Regenerate the GGUF without tensors the encoder cannot consume"],"exampleFix":"# before\nRuntimeError: Unexpected keys loading Gemma-2 GGUF encoder: ['layers.0.self_attn.kv_proj.weight']\n# after\n_GEMMA_GGUF_KEY_MAP = {..., \"attn_kv\": \"self_attn.k_proj\"}  # key now matches Gemma2Model","handlingStrategy":"try-catch","validationCode":"def keys_match_model(converted_keys, model):\n    model_keys = set(model.state_dict().keys())\n    unexpected = [k for k in converted_keys if k not in model_keys]\n    return not unexpected, unexpected","typeGuard":"def is_valid_state_dict(sd: dict) -> bool:\n    model_keys = set(Gemma2Model(gemma_config).state_dict().keys())\n    return all(k in model_keys for k in sd)","tryCatchPattern":"try:\n    model = load_gemma2_model_from_gguf(gguf_path, dtype)\nexcept RuntimeError as e:\n    if \"Unexpected keys loading Gemma-2 GGUF encoder\" in str(e):\n        print(f\"Converter/model key mismatch: {e}; update the key map or transformers version\")\n    else:\n        raise","preventionTips":["Pin the transformers version the key map was written against","Test converted state dicts against model.state_dict() keys in CI","Re-run conversion tests when upgrading transformers"],"tags":["gguf","state-dict","gemma2","transformers-version"],"backgroundTag":"state-dict-key-mismatch","analyzedSha":"0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06","analyzedAt":"2026-08-29T04:46:49.967Z","schemaVersion":2},"datasetVersion":"2026-08-29T07:17:48.351Z"}