{"record":{"id":"e7455186416cbf1b","repo":"invoke-ai/InvokeAI","slug":"gemma-2-gguf-encoder-has-parameters-left-on-the-me","errorCode":null,"errorMessage":"Gemma-2 GGUF encoder has parameters left on the meta device after loading: {meta_params[:10]}","messagePattern":"Gemma-2 GGUF encoder has parameters left on the meta device after loading: (.+?)","errorType":"validation","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"invokeai/backend/model_manager/load/model_loaders/gemma2_encoder.py","lineNumber":200,"sourceCode":"            if not isinstance(param, GGMLTensor):\n                continue\n            if isinstance(module, torch.nn.Embedding):\n                setattr(module, name, torch.nn.Parameter(param.get_dequantized_tensor(), requires_grad=False))\n            elif param.ndim == 1:\n                setattr(module, name, torch.nn.Parameter(param.get_dequantized_tensor() - 1.0, requires_grad=False))\n\n    # Re-materialize meta buffers not present in the GGUF (the rotary embedding's inv_freq).\n    for name, buf in list(model.named_buffers()):\n        if buf.is_meta and name.endswith(\"inv_freq\"):\n            head_dim = gemma_config.head_dim\n            base = float(getattr(gemma_config, \"rope_theta\", 10000.0))\n            inv_freq = 1.0 / (base ** (torch.arange(0, head_dim, 2, dtype=torch.float32) / head_dim))\n            parent = model.get_submodule(name.rsplit(\".\", 1)[0]) if \".\" in name else model\n            parent.register_buffer(name.rsplit(\".\", 1)[-1], inv_freq.to(compute_dtype), persistent=False)\n\n    meta_params = [n for n, p in model.named_parameters() if p.is_meta]\n    if meta_params:\n        raise RuntimeError(\n            f\"Gemma-2 GGUF encoder has parameters left on the meta device after loading: {meta_params[:10]}\"\n        )\n\n    model.eval()\n    return model\n","sourceCodeStart":182,"sourceCodeEnd":206,"githubUrl":"https://github.com/invoke-ai/InvokeAI/blob/0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06/invokeai/backend/model_manager/load/model_loaders/gemma2_encoder.py#L182-L206","documentation":"After load_state_dict(..., assign=True), any parameter still on the meta device means the state dict never provided a tensor for it. The loader re-registers RoPE inv_freq buffers but parameters cannot stay meta, so it raises a RuntimeError listing up to 10 such parameter names.","triggerScenarios":"A converted state dict missing weights the model expects — e.g. the GGUF lacks a tensor the converter should map (missing attn/ffn component or embedding), or conversion silently drops keys; also mismatched transformers Gemma2Model module structure.","commonSituations":"Truncated or corrupted GGUF files; converter map missing an entry so the corresponding model param stays meta; transformers version with modules the GGUF has no tensors for; missing output_norm in an encoder-exported GGUF.","solutions":["Check the listed meta parameter names and ensure _GEMMA_GGUF_KEY_MAP maps the corresponding GGUF tensors","Regenerate/re-download the GGUF — it may be truncated or missing tensors","Verify all non-buffer parameters get tensors after conversion (add explicit checks in the converter)"],"exampleFix":"# before\nRuntimeError: Gemma-2 GGUF encoder has parameters left on the meta device after loading: ['layers.5.mlp.gate_proj.weight']\n# after\n_GEMMA_GGUF_KEY_MAP = {..., \"ffn_gate\": \"mlp.gate_proj\"}  # tensor now supplied","handlingStrategy":"validation","validationCode":"def all_params_materialized(model):\n    meta = [n for n, p in model.named_parameters() if p.is_meta]\n    return not meta, meta","typeGuard":"def is_fully_loaded(model) -> bool:\n    return not any(p.is_meta for p in model.parameters())","tryCatchPattern":"try:\n    model = load_gemma2_model_from_gguf(gguf_path, dtype)\nexcept RuntimeError as e:\n    if \"parameters left on the meta device\" in str(e):\n        print(f\"GGUF missing tensors: {e}; re-download or fix key map\")\n    else:\n        raise","preventionTips":["Verify GGUF file integrity (size/checksum) after download","Ensure every model parameter has a corresponding mapped GGUF tensor","Run a forward pass on a dummy input after loading to catch missing weights"],"tags":["gguf","meta-device","state-dict","gemma2"],"backgroundTag":"uninitialized-meta-parameter","analyzedSha":"0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06","analyzedAt":"2026-08-29T04:46:49.967Z","schemaVersion":2},"datasetVersion":"2026-08-29T07:17:48.351Z"}