sgl-project/sglang · error · ValueError

Unsupported Comfy NVFP4 companion for {prefix!r}: {marker_fo

Error message

Unsupported Comfy NVFP4 companion for {prefix!r}: {marker_format!r}

What it means

ComfyNvfp4Config validates companion layer markers: int8_tensorwise markers are allowed only when marked _is_rowwise; every other marker must have format "nvfp4". Any other companion format raises with the offending prefix and format.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/comfy_nvfp4.py:184

    """Dispatch full-precision Comfy NVFP4 linears and their INT8 embedding."""

    checkpoint_uses_comfy_quantization = True

    def __init__(self, layer_markers: dict[str, dict[str, Any]]) -> None:
        super().__init__(
            is_checkpoint_nvfp4_serialized=True,
            group_size=16,
            exclude_modules=[],
            checkpoint_uses_comfy_quantization=True,
        )
        self.layer_markers = layer_markers
        self.selected: list[str] = []
        for prefix, marker in layer_markers.items():
            marker_format = marker.get("format")
            if marker_format == "int8_tensorwise" and marker.get("_is_rowwise"):
                continue
            if marker_format != "nvfp4":
                raise ValueError(
                    f"Unsupported Comfy NVFP4 companion for {prefix!r}: "
                    f"{marker_format!r}"
                )
            if marker.get("full_precision_matrix_mult") is not True:
                raise ValueError(
                    f"Comfy NVFP4 layer {prefix!r} must request "
                    "full_precision_matrix_mult"
                )

    @classmethod
    def get_name(cls) -> str:
        return "comfy_nvfp4"

    @classmethod
    def get_supported_act_dtypes(cls) -> list[torch.dtype]:
        return [torch.bfloat16, torch.float16]

    @classmethod

View on GitHub (pinned to 0132848349)

Solutions

1. Inspect the reported prefix's safetensors marker and re-quantize that layer to nvfp4 (or int8_tensorwise with _is_rowwise)
2. Export the checkpoint with only supported companion formats
3. Use the config class matching the actual formats

for p, m in layer_markers.items():
    fmt = m.get("format")
    ok = fmt == "nvfp4" or (fmt == "int8_tensorwise" and m.get("_is_rowwise"))
    if not ok:
        raise SystemExit(f"unsupported companion {p}: {fmt}")

When it happens

Trigger: Constructing ComfyNvfp4Config with layer_markers containing a companion whose format is neither nvfp4 nor the allowed int8_tensorwise+_is_rowwise combination (e.g. fp8, plain int8_tensorwise without rowwise flag).

Common situations: Mixed-precision Comfy checkpoints combining nvfp4 with other quant formats; checkpoint metadata saved by an incompatible ComfyUI version.

Related errors

• Unsupported Comfy FP8 layer formats: {unsupported}
• Comfy full_precision_matrix_mult does not support fused line
• Comfy NVFP4 layer {prefix!r} must request full_precision_mat
• comfy_nvfp4 is inferred from per-layer checkpoint metadata;
• MIXED_PRECISION checkpoint has no NVFP4 layers to requantize

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/582a10684cb936c6. Report an issue: GitHub.