{"record":{"id":"f568004d72e8a840","repo":"sgl-project/sglang","slug":"gguf-models-are-not-supported","errorCode":null,"errorMessage":"GGUF models are not supported.","messagePattern":"GGUF models are not supported\\.","errorType":"validation","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/utils/hf_diffusers_utils.py","lineNumber":431,"sourceCode":"        AutoConfig.register(name, cls)\n\n\ndef download_from_hf(model_path: str):\n    if os.path.exists(model_path):\n        return model_path\n\n    return snapshot_download(model_path, allow_patterns=[\"*.json\", \"*.bin\", \"*.model\"])\n\n\ndef get_hf_config(\n    component_model_path: str,\n    trust_remote_code: bool,\n    revision: str | None = None,\n    model_override_args: dict | None = None,\n    **kwargs,\n) -> PretrainedConfig:\n    if check_gguf_file(component_model_path):\n        raise NotImplementedError(\"GGUF models are not supported.\")\n\n    config = AutoConfig.from_pretrained(\n        component_model_path,\n        trust_remote_code=trust_remote_code,\n        revision=revision,\n        **kwargs,\n    )\n    if config.model_type in _CONFIG_REGISTRY:\n        config_class = _CONFIG_REGISTRY[config.model_type]\n        config = config_class.from_pretrained(component_model_path, revision=revision)\n        # NOTE(HandH1998): Qwen2VL requires `_name_or_path` attribute in `config`.\n        config._name_or_path = component_model_path\n    if model_override_args:\n        config.update(model_override_args)\n\n    return config\n\n","sourceCodeStart":413,"sourceCodeEnd":449,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/utils/hf_diffusers_utils.py#L413-L449","documentation":"get_hf_config refuses to load configs from GGUF-quantized checkpoints; only standard HF/diffusers formats are supported. It detects a GGUF file via check_gguf_file on the model path and raises NotImplementedError immediately.","triggerScenarios":"Passing a GGUF-quantized model path (e.g. something quantized with llama.cpp tooling) to load_native, load_customized, or create_kt_config_from_server_args, all of which route through get_hf_config.","commonSituations":"User points --model at a local GGUF file or a repo containing *.gguf weights to save memory; mixing llama.cpp artifacts into a diffusers/HF pipeline workflow.","solutions":["Use the original HF or diffusers-format model instead of the GGUF quantization","If quantization is required, use a supported format (e.g. bitsandnames/awq via the library's own quant path)"],"exampleFix":"# before\n--model /models/flux-dev-Q4_K_M.gguf\n# after\n--model black-forest-labs/FLUX.1-dev","handlingStrategy":"fallback","validationCode":"from pathlib import Path\nis_gguf = any(p.suffix == '.gguf' for p in Path(model_path).glob('*.gguf')) if Path(model_path).is_dir() else model_path.endswith('.gguf')","typeGuard":null,"tryCatchPattern":"try:\n    cfg = get_hf_config(path, ...)\nexcept NotImplementedError:\n    cfg = get_hf_config(fallback_hf_path, ...)","preventionTips":["Keep a non-GGUF copy of models for this pipeline","Block GGUF paths in config validation"],"tags":["model-format","gguf","unsupported"],"backgroundTag":"unsupported-model-format","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}