{"record":{"id":"c3eea3244be8e114","repo":"sgl-project/sglang","slug":"quantoint8config-must-be-constructed-from-safetens","errorCode":null,"errorMessage":"QuantoInt8Config must be constructed from safetensors metadata","messagePattern":"QuantoInt8Config must be constructed from safetensors metadata","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py","lineNumber":63,"sourceCode":"    @classmethod\n    def get_name(cls) -> str:\n        return \"quanto_int8\"\n\n    @classmethod\n    def get_supported_act_dtypes(cls) -> list[torch.dtype]:\n        return [torch.bfloat16, torch.float16]\n\n    @classmethod\n    def get_min_capability(cls) -> int:\n        return 0\n\n    @staticmethod\n    def get_config_filenames() -> list[str]:\n        return []\n\n    @classmethod\n    def from_config(cls, config: dict[str, Any]) -> QuantoInt8Config:\n        raise ValueError(\n            \"QuantoInt8Config must be constructed from safetensors metadata\"\n        )\n\n    def get_quant_method(\n        self, layer: torch.nn.Module, prefix: str\n    ) -> QuantizeMethodBase | None:\n        if isinstance(layer, DiffusionLinearBase):\n            unquantized_method = DiffusionUnquantizedLinearMethod\n        elif isinstance(layer, SrtLinearBase):\n            unquantized_method = SrtUnquantizedLinearMethod\n        else:\n            return None\n        if prefix not in self.layer_prefixes:\n            return unquantized_method()\n        self.selected.add(prefix)\n        return QuantoInt8LinearMethod()\n\n","sourceCodeStart":45,"sourceCodeEnd":81,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/configs/quanto_int8_config.py#L45-L81","documentation":"QuantoInt8Config.from_config is intentionally disabled (raises always) and get_config_filenames returns []. Quanto int8 metadata lives in safetensors headers, not a JSON config file, so the config must be built via inspect_quanto_int8_checkpoint instead of the standard from_config path.","triggerScenarios":"Any code path (e.g. a generic loader calling QuantizationConfig.from_config(config_dict)) that tries to deserialize QuantoInt8Config from a hf quantization_config dict.","commonSituations":"Plugging QuantoInt8Config into a framework that auto-instantiates configs via from_config; forgetting to special-case quanto detection before the generic loader.","solutions":["Call inspect_quanto_int8_checkpoint(safetensors_file) to build the config from checkpoint metadata","Special-case quanto checkpoints in your loader before falling back to from_config","Check checkpoint metadata['quantization_format'] == 'quanto' first"],"exampleFix":"# before\ncfg = QuantoInt8Config.from_config({})  # always raises\n\n# after\nfrom safetensors import safe_open\nwith safe_open(path, framework=\"pt\") as f:\n    cfg = inspect_quanto_int8_checkpoint(f)","handlingStrategy":"fallback","validationCode":"from safetensors import safe_open\nwith safe_open(ckpt_path, framework=\"pt\") as f:\n    if (f.metadata() or {}).get(\"quantization_format\") == \"quanto\":\n        cfg = inspect_quanto_int8_checkpoint(f)\n    else:\n        cfg = SomeOtherConfig.from_config(config_dict)","typeGuard":null,"tryCatchPattern":"try:\n    cfg = QuantoInt8Config.from_config(d)\nexcept ValueError:\n    cfg = inspect_quanto_int8_checkpoint(open_safetensors(path))","preventionTips":["Route quanto checkpoints through inspect_quanto_int8_checkpoint, never from_config","Check metadata['quantization_format'] before choosing a config constructor"],"tags":["quantization","quanto","config-loading"],"backgroundTag":"unsupported-construction-path","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}