{"record":{"id":"eb84c7880ce85588","repo":"sgl-project/sglang","slug":"found-unknown-quantization-quantization-in-con","errorCode":null,"errorMessage":"Found unknown quantization='{quantization}' in config","messagePattern":"Found unknown quantization='(.+?)' in config","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/utils/hf_transformers/mistral_utils.py","lineNumber":237,"sourceCode":"\n\ndef _remap_mistral_quantization_args(config: dict) -> dict:\n    if config.get(\"quantization\"):\n        quantization = config.pop(\"quantization\", {})\n        if quantization.get(\"qformat_weight\") == \"fp8_e4m3\":\n            qscheme_act = quantization.get(\"qscheme_act\")\n            assert qscheme_act in (\n                \"NO_SCALES\",\n                \"TENSOR\",\n                None,\n            ), \"Only NO_SCALES and TENSOR (default) are supported for qscheme_act\"\n            is_dynamic = qscheme_act == \"NO_SCALES\"\n            config[\"quantization_config\"] = {\n                \"quant_method\": \"fp8\",\n                \"activation_scheme\": \"dynamic\" if is_dynamic else \"static\",\n            }\n        else:\n            raise ValueError(f\"Found unknown quantization='{quantization}' in config\")\n\n    return config\n\n\ndef _remap_mistral_audio_args(config: dict) -> dict:\n    whisper_args = config[\"multimodal\"].pop(\"whisper_model_args\")\n    encoder_args = whisper_args[\"encoder_args\"]\n    downsample_args = whisper_args[\"downsample_args\"]\n\n    quant_config = config.get(\"quantization_config\")\n    config = {\n        \"model_type\": \"whixtral\",\n        \"architectures\": [\"VoxtralForConditionalGeneration\"],\n        \"text_config\": PretrainedConfig.from_dict(config),\n        \"audio_config\": WhisperConfig(\n            num_mel_bins=encoder_args[\"audio_encoding_args\"][\"num_mel_bins\"],\n            window_size=encoder_args[\"audio_encoding_args\"][\"window_size\"],\n            sampling_rate=encoder_args[\"audio_encoding_args\"][\"sampling_rate\"],","sourceCodeStart":219,"sourceCodeEnd":255,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/utils/hf_transformers/mistral_utils.py#L219-L255","documentation":"While adapting a Mistral-format params.json config to HF style, the quantization field value does not match any known scheme (fp8 dynamic/static, etc.), so it cannot be remapped into quantization_config.","triggerScenarios":"Loading a Mistral-format model whose params.json declares a new or unrecognized quantization string (e.g. a newer int4/fp4 variant).","commonSituations":"Mistral releases a new quant format before sglang adds a remapping; custom-quantized Mistral models.","solutions":["Upgrade sglang to a version supporting the quant scheme","Serve the HF-format version of the model instead of Mistral format","Remove/bypass the quantization entry only if the weights are actually unquantized"],"exampleFix":null,"handlingStrategy":"fallback","validationCode":"q = params.get('quantization'); known = {None,'fp8','fp8_dynamic', ...}\nif q not in known: prefer HF-format model","typeGuard":null,"tryCatchPattern":"try:\n    adapt_config_dict(cfg)\nexcept ValueError as e:\n    if 'unknown quantization' in str(e): use_hf_checkpoint()","preventionTips":["Serve new Mistral quants from HF-format checkpoints until sglang adds support"],"tags":["mistral","quantization","config"],"backgroundTag":"unsupported-quantization-scheme","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}