invoke-ai/InvokeAI · critical · RuntimeError

Gemma-2 GGUF encoder has parameters left on the meta device

Error message

Gemma-2 GGUF encoder has parameters left on the meta device after loading: {meta_params[:10]}

What it means

After load_state_dict(..., assign=True), any parameter still on the meta device means the state dict never provided a tensor for it. The loader re-registers RoPE inv_freq buffers but parameters cannot stay meta, so it raises a RuntimeError listing up to 10 such parameter names.

Source

Thrown at invokeai/backend/model_manager/load/model_loaders/gemma2_encoder.py:200

            if not isinstance(param, GGMLTensor):
                continue
            if isinstance(module, torch.nn.Embedding):
                setattr(module, name, torch.nn.Parameter(param.get_dequantized_tensor(), requires_grad=False))
            elif param.ndim == 1:
                setattr(module, name, torch.nn.Parameter(param.get_dequantized_tensor() - 1.0, requires_grad=False))

    # Re-materialize meta buffers not present in the GGUF (the rotary embedding's inv_freq).
    for name, buf in list(model.named_buffers()):
        if buf.is_meta and name.endswith("inv_freq"):
            head_dim = gemma_config.head_dim
            base = float(getattr(gemma_config, "rope_theta", 10000.0))
            inv_freq = 1.0 / (base ** (torch.arange(0, head_dim, 2, dtype=torch.float32) / head_dim))
            parent = model.get_submodule(name.rsplit(".", 1)[0]) if "." in name else model
            parent.register_buffer(name.rsplit(".", 1)[-1], inv_freq.to(compute_dtype), persistent=False)

    meta_params = [n for n, p in model.named_parameters() if p.is_meta]
    if meta_params:
        raise RuntimeError(
            f"Gemma-2 GGUF encoder has parameters left on the meta device after loading: {meta_params[:10]}"
        )

    model.eval()
    return model

View on GitHub (pinned to 0b6a024f2f)

Solutions

1. Check the listed meta parameter names and ensure _GEMMA_GGUF_KEY_MAP maps the corresponding GGUF tensors
2. Regenerate/re-download the GGUF — it may be truncated or missing tensors
3. Verify all non-buffer parameters get tensors after conversion (add explicit checks in the converter)

Example fix

# before
RuntimeError: Gemma-2 GGUF encoder has parameters left on the meta device after loading: ['layers.5.mlp.gate_proj.weight']
# after
_GEMMA_GGUF_KEY_MAP = {..., "ffn_gate": "mlp.gate_proj"}  # tensor now supplied

def all_params_materialized(model):
    meta = [n for n, p in model.named_parameters() if p.is_meta]
    return not meta, meta

def is_fully_loaded(model) -> bool:
    return not any(p.is_meta for p in model.parameters())

try:
    model = load_gemma2_model_from_gguf(gguf_path, dtype)
except RuntimeError as e:
    if "parameters left on the meta device" in str(e):
        print(f"GGUF missing tensors: {e}; re-download or fix key map")
    else:
        raise

When it happens

Trigger: A converted state dict missing weights the model expects — e.g. the GGUF lacks a tensor the converter should map (missing attn/ffn component or embedding), or conversion silently drops keys; also mismatched transformers Gemma2Model module structure.

Common situations: Truncated or corrupted GGUF files; converter map missing an entry so the corresponding model param stays meta; transformers version with modules the GGUF has no tensors for; missing output_norm in an encoder-exported GGUF.

Related errors

• Unexpected keys loading Gemma-2 GGUF encoder: {unexpected[:1
• Gemma2 GGUF embedding_length {hidden_size} is incompatible w
• state dict does not look like GGUF quantized
• state dict looks like GGUF quantized
• state dict does not look like a T5 encoder (no 'enc.blk.*' k

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/e7455186416cbf1b. Report an issue: GitHub.