{"record":{"id":"d41d82ceaf01a3d8","repo":"sgl-project/sglang","slug":"gguf-diffusion-checkpoints-are-incompatible-with-f","errorCode":null,"errorMessage":"GGUF diffusion checkpoints are incompatible with FSDP inference. Run without --use-fsdp-inference, or keep this component offloaded so FSDP does not manage it.","messagePattern":"GGUF diffusion checkpoints are incompatible with FSDP inference\\. Run without --use-fsdp-inference, or keep this component offloaded so FSDP does not manage it\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/loader/transformer_load_utils.py","lineNumber":556,"sourceCode":"    if server_args.nunchaku_config is not None:\n        raise ValueError(\n            \"--enable-svdquant cannot be combined with a GGUF transformer: both \"\n            \"supply the transformer weights. Point \"\n            \"--transformer-weights-path at either an SVDQuant checkpoint or a \"\n            \".gguf, not one while requesting the other.\"\n        )\n    if not current_platform.is_cuda():\n        raise ValueError(\n            \"GGUF diffusion checkpoints require CUDA; the GGML kernels have no \"\n            f\"{current_platform.device_type} implementation.\"\n        )\n    uses_fsdp = (\n        server_args.should_use_fsdp_for_component(component_name)\n        if component_name is not None\n        else server_args.use_fsdp_inference\n    )\n    if uses_fsdp:\n        raise ValueError(\n            \"GGUF diffusion checkpoints are incompatible with FSDP inference. \"\n            \"Run without --use-fsdp-inference, or keep this component offloaded \"\n            \"so FSDP does not manage it.\"\n        )\n    if server_args.lora_path is not None:\n        raise ValueError(\n            \"LoRA is not supported on a GGUF transformer: an adapter cannot be \"\n            \"merged into packed GGML blocks. Use the unquantized checkpoint to \"\n            \"serve LoRA.\"\n        )\n    # H3's AdaLN paths read the transformer's safetensors directly -- the cache\n    # builder needs unquantized weights, and the online rebuild is handed the\n    # safetensors file list, which is empty for a GGUF load.\n    if server_args.minimax_h3_adaln_online:\n        raise ValueError(\n            \"--minimax-h3-adaln-online rebuilds AdaLN outputs from the \"\n            \"safetensors checkpoint and cannot read a GGUF transformer.\"\n        )","sourceCodeStart":538,"sourceCodeEnd":574,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/loader/transformer_load_utils.py#L538-L574","documentation":"GGUF diffusion checkpoints cannot be sharded/managed by FSDP because the packed GGML blocks don't fit FSDP's parameter expectations, so the combination is refused.","triggerScenarios":"server_args.use_fsdp_inference is True, or should_use_fsdp_for_component(component_name) returns True for the component being loaded, while loading a GGUF transformer.","commonSituations":"Enabling --use-fsdp-inference for multi-GPU memory sharding and then pointing at a .gguf checkpoint; or FSDP picking up the transformer component because it wasn't offloaded/excluded.","solutions":["Run without --use-fsdp-inference for this GGUF model","Keep the transformer component offloaded/excluded so FSDP does not manage it","Use an FSDP-compatible safetensors checkpoint if FSDP is required"],"exampleFix":"# before\n--use-fsdp-inference --transformer-weights-path model.gguf\n# after\n--transformer-weights-path model.gguf","handlingStrategy":"validation","validationCode":"if str(server_args.transformer_weights_path or '').endswith('.gguf') and (server_args.use_fsdp_inference or server_args.should_use_fsdp_for_component('transformer')):\n    raise SystemExit('GGUF transformer is incompatible with FSDP inference')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Don't enable --use-fsdp-inference with GGUF checkpoints","Offload the transformer component so FSDP skips it"],"tags":["gguf","fsdp","distributed","config-conflict"],"backgroundTag":"incompatible-launch-flags","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}