sgl-project/sglang · error · ValueError

comfy_nvfp4 is inferred from per-layer checkpoint metadata;

Error message

comfy_nvfp4 is inferred from per-layer checkpoint metadata; it is not an online quantization method

What it means

ComfyNvfp4Config is derived from per-layer checkpoint metadata, not from a config file or online (on-the-fly) quantization. from_config deliberately raises to block using comfy_nvfp4 as an online quantization method name.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/comfy_nvfp4.py:212

    @classmethod
    def get_name(cls) -> str:
        return "comfy_nvfp4"

    @classmethod
    def get_supported_act_dtypes(cls) -> list[torch.dtype]:
        return [torch.bfloat16, torch.float16]

    @classmethod
    def get_min_capability(cls) -> int:
        return 0

    @classmethod
    def get_config_filenames(cls) -> list[str]:
        return []

    @classmethod
    def from_config(cls, config: dict[str, Any]) -> ComfyNvfp4Config:
        raise ValueError(
            "comfy_nvfp4 is inferred from per-layer checkpoint metadata; "
            "it is not an online quantization method"
        )

    def get_quant_method(
        self, layer: nn.Module, prefix: str
    ) -> QuantizeMethodBase | None:
        marker = self.layer_markers.get(prefix)
        if isinstance(layer, VocabParallelEmbedding):
            if marker is None:
                return None
            if marker.get("format") != "int8_tensorwise" or not marker.get(
                "_is_rowwise"
            ):
                raise ValueError(
                    f"Unsupported quantized embedding marker for {prefix!r}: {marker}"
                )
            self.selected.append(prefix)

View on GitHub (pinned to 0132848349)

Solutions

1. Remove the explicit quantization setting and let the loader infer comfy_nvfp4 from the checkpoint's per-layer markers
2. Quantize the model offline with a tool that emits nvfp4 markers, then load it
3. Use fp8 or another supported online quantization method instead

from ...comfy_nvfp4 import ComfyNvfp4Config
if ComfyNvfp4Config.get_config_filenames() == []:
    # marker-inferred method; do not pass as an online quantization name

When it happens

Trigger: Passing quantization="comfy_nvfp4" on a CLI/config path that resolves it via from_config, or calling ComfyNvfp4Config.from_config(...) directly — i.e. trying to quantize an unquantized model online.

Common situations: Users adding --quantization comfy_nvfp4 expecting online nvfp4 quantization of a bf16 checkpoint; generic loaders that call from_config for all methods.

Related errors

• ComfyFp8Config must be constructed from safetensors layer ma
• Comfy full_precision_matrix_mult does not support fused line
• Unsupported Comfy NVFP4 companion for {prefix!r}: {marker_fo
• Comfy NVFP4 layer {prefix!r} must request full_precision_mat
• Type must match: {self.a_dtype} != {self.b_dtype}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ec23b7de57adf409. Report an issue: GitHub.