{"record":{"id":"061e1ce5c21e7fe2","repo":"sgl-project/sglang","slug":"gguf-diffusion-checkpoints-require-cuda-the-ggml","errorCode":null,"errorMessage":"GGUF diffusion checkpoints require CUDA; the GGML kernels have no {current_platform.device_type} implementation.","messagePattern":"GGUF diffusion checkpoints require CUDA; the GGML kernels have no (.+?) implementation\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/loader/transformer_load_utils.py","lineNumber":546,"sourceCode":"        )\n    if server_args.quantization is not None:\n        raise ValueError(\n            f\"--quantization {server_args.quantization} cannot be combined with \"\n            \"a GGUF transformer, whose quantization is fixed by the checkpoint. \"\n            \"Drop the flag, or use an unquantized checkpoint to quantize online.\"\n        )\n    # Nunchaku shares --transformer-weights-path with GGUF, and the GGUF plan is\n    # resolved first, so without this the SVDQuant request would be dropped in\n    # silence rather than refused.\n    if server_args.nunchaku_config is not None:\n        raise ValueError(\n            \"--enable-svdquant cannot be combined with a GGUF transformer: both \"\n            \"supply the transformer weights. Point \"\n            \"--transformer-weights-path at either an SVDQuant checkpoint or a \"\n            \".gguf, not one while requesting the other.\"\n        )\n    if not current_platform.is_cuda():\n        raise ValueError(\n            \"GGUF diffusion checkpoints require CUDA; the GGML kernels have no \"\n            f\"{current_platform.device_type} implementation.\"\n        )\n    uses_fsdp = (\n        server_args.should_use_fsdp_for_component(component_name)\n        if component_name is not None\n        else server_args.use_fsdp_inference\n    )\n    if uses_fsdp:\n        raise ValueError(\n            \"GGUF diffusion checkpoints are incompatible with FSDP inference. \"\n            \"Run without --use-fsdp-inference, or keep this component offloaded \"\n            \"so FSDP does not manage it.\"\n        )\n    if server_args.lora_path is not None:\n        raise ValueError(\n            \"LoRA is not supported on a GGUF transformer: an adapter cannot be \"\n            \"merged into packed GGML blocks. Use the unquantized checkpoint to \"","sourceCodeStart":528,"sourceCodeEnd":564,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/loader/transformer_load_utils.py#L528-L564","documentation":"The GGML dequantization kernels used to load GGUF diffusion checkpoints only have CUDA implementations, so loading a GGUF transformer on any non-CUDA platform is rejected up front.","triggerScenarios":"current_platform.is_cuda() is False (e.g. ROCm/HIP, CPU, Metal) while a .gguf transformer is requested via --transformer-weights-path.","commonSituations":"Running on AMD GPUs without CUDA support, CPU-only dev boxes, or a container without a visible CUDA device.","solutions":["Run on an NVIDIA CUDA machine","Otherwise use a safetensors checkpoint compatible with your platform instead of GGUF"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"import torch\nif not torch.cuda.is_available() and str(server_args.transformer_weights_path or '').endswith('.gguf'):\n    raise SystemExit('GGUF transformer requires CUDA')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Gate GGUF checkpoints on torch.cuda.is_available() in launch tooling"],"tags":["gguf","cuda","platform-unsupported"],"backgroundTag":"unsupported-device-platform","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}