{"record":{"id":"069569e00479f268","repo":"sgl-project/sglang","slug":"ggufconfig-must-be-constructed-from-a-gguf-checkpo","errorCode":null,"errorMessage":"GGUFConfig must be constructed from a GGUF checkpoint","messagePattern":"GGUFConfig must be constructed from a GGUF checkpoint","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/gguf.py","lineNumber":80,"sourceCode":"    @classmethod\n    def get_name(cls) -> str:\n        return \"gguf\"\n\n    @classmethod\n    def get_supported_act_dtypes(cls) -> list[torch.dtype]:\n        return [torch.float32, torch.float16, torch.bfloat16]\n\n    @classmethod\n    def get_min_capability(cls) -> int:\n        return 60\n\n    @staticmethod\n    def get_config_filenames() -> list[str]:\n        return []\n\n    @classmethod\n    def from_config(cls, config: dict[str, Any]) -> GGUFConfig:\n        raise ValueError(\"GGUFConfig must be constructed from a GGUF checkpoint\")\n\n    def get_quant_method(\n        self, layer: nn.Module, prefix: str\n    ) -> QuantizeMethodBase | None:\n        if isinstance(layer, LinearBase):\n            unquantized_method = UnquantizedLinearMethod\n        elif isinstance(layer, VocabParallelEmbedding):\n            unquantized_method = None\n        else:\n            return None\n\n        metadata = self.tensor_meta.get(f\"{prefix}.weight\")\n        if metadata is None:\n            raise ValueError(\n                f\"Linear layer {prefix!r} has no weight in the GGUF checkpoint \"\n                f\"{self.gguf_file!r}\"\n            )\n        weight_type = metadata.weight_type","sourceCodeStart":62,"sourceCodeEnd":98,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/gguf.py#L62-L98","documentation":"GGUFConfig.from_config unconditionally raises because GGUF checkpoints are not loaded through the standard quantization-config JSON path. GGUF models are constructed directly from the .gguf file, so building the config from a plain dict is unsupported.","triggerScenarios":"Calling GGUFConfig.from_config(config_dict), or a generic config loader auto-detecting a quant method named GGUF in hf_quant_config.json and trying to instantiate it via from_config.","commonSituations":"Passing a GGUF file through a loader designed for HuggingFace quantization configs (e.g. transformers-style CLI), or misconfigured quantization: gguf in a config JSON.","solutions":["Load GGUF models via the dedicated GGUF loading path (construct GGUFConfig directly from the .gguf checkpoint)","Remove/fix the quantization config that routes to GGUFConfig.from_config","Convert the checkpoint to a supported format if GGUF loading is not intended"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"if isinstance(config, dict) and 'gguf' in str(config.get('quant_method','')).lower():\n    raise SystemExit('Load GGUF models via the GGUF loading path, not from_config')","typeGuard":"def is_gguf_checkpoint(path: str) -> bool:\n    return path.endswith('.gguf')","tryCatchPattern":null,"preventionTips":["Detect .gguf files early and route them to the dedicated GGUF loader","Never put gguf in a HuggingFace-style quantization config JSON"],"tags":["gguf","quantization","config","unsupported-operation"],"backgroundTag":"unsupported-config-path","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}