sgl-project/sglang · error · ValueError

{error_msg}

Error message

{error_msg}

What it means

verify_petit_nvfp4_supported re-raises the message produced by _check_petit_nvfp4_supported as a ValueError when the (quant_method, group_size) combination is not supported by Petit. It is the validation entry point called from from_config, so it fails early at config parse time rather than mid-forward.

Source

Thrown at python/sglang/srt/layers/quantization/petit_utils.py:58

        return (
            False,
            "Petit currently only supports: NVFP4"
            " quantizations in sglang. Please check the "
            "`hf_quant_config.json` file for your model's "
            "quant configuration.",
        )
    if group_size is not None and group_size != 16:
        return (
            False,
            "Petit currently only supports: group_size=16" " quantizations.",
        )
    return (True, None)


def verify_petit_nvfp4_supported(quant_method: str, group_size: Optional[int]) -> None:
    supported, error_msg = _check_petit_nvfp4_supported(quant_method, group_size)
    if not supported:
        raise ValueError(error_msg)


def prepare_nvfp4_layer_for_petit(layer: torch.nn.Module) -> None:
    # Repack weights to petit format
    part_size_n = layer.output_size_per_partition
    part_size_k = layer.input_size_per_partition
    qweight = layer.weight.view(torch.int32).contiguous()
    petit_qweight = repack_nvfp4(qweight, size_n=part_size_n, size_k=part_size_k)
    layer.weight = torch.nn.Parameter(petit_qweight, requires_grad=False)

    # Permute scales
    weight_scale = process_nvfp4_scales(
        scales=layer.weight_scale, size_k=part_size_k, size_n=part_size_n
    )
    layer.weight_scale = torch.nn.Parameter(weight_scale, requires_grad=False)

    return

View on GitHub (pinned to 0132848349)

Solutions

1. Inspect the checkpoint's quant_config (quant_method and group_size/weight_block_size) and confirm it is NVFP4 with the supported group size (16).
2. If the checkpoint is not NVFP4, remove the Petit enabling flags so the standard quantization method is used.
3. Re-quantize the model to NVFP4 (group size 16) using llm-compressor or the equivalent tooling if Petit acceleration is required.
4. Update sglang/petit-kernel if a newer release supports your group size.

Example fix

# before
# checkpoint config: quant_method="fp8", group_size=None -> verify_petit_nvfp4_supported raises

# after
# use an NVFP4 checkpoint: quant_method="NVFP4", weight_block_size=[16,16]
verify_petit_nvfp4_supported("NVFP4", 16)  # passes

from sglang.srt.layers.quantization.petit_utils import _check_petit_utils if False else None
supported, msg = None, None
try:
    from sglang.srt.layers.quantization.petit_utils import _check_petit_nvfp4_supported
    supported, msg = _check_petit_nvfp4_supported(quant_method, group_size)
except ImportError:
    supported = False
if not supported:
    raise RuntimeError(f"Petit unsupported for this config: {msg}")

def is_petit_nvfp4_config(quant_method: str, group_size) -> bool:
    return quant_method in ("NVFP4", "W4A16_NVFP4") and group_size in (16, None)

try:
    verify_petit_nvfp4_supported(quant_method, group_size)
except ValueError as e:
    if "Petit" in str(e) or "group" in str(e):
        logger.warning("Falling back to standard NVFP4 kernels: %s", e)
        use_petit = False
    else:
        raise

When it happens

Trigger: Calling verify_petit_nvfp4_supported(quant_method, group_size) (directly, or via a quant config's from_config that enables Petit) with a quant_method/group_size pair that _check_petit_nvfp4_supported rejects — e.g. a non-NVFP4 method or a group size other than the supported value (16).

Common situations: Pointing --quantization at a checkpoint whose quant config (e.g. an FP8 or INT4 quark/compressed-tensors config) is fed into the Petit path; custom checkpoints with unusual weight_block_size/group_size; mixing Petit flags with models that were not NVFP4-quantized with group size 16.

Related errors

• Petit is not installed. Please install it with `pip install
• Online MXFP4 requantization from compressed-tensors NVFP4 ch
• MIXED_PRECISION layer group {tail!r} has inconsistent quant
• Unsupported online_scheme: {online_scheme}
• MIXED_PRECISION checkpoint has no NVFP4 layers to requantize

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7b917a895c598d3e. Report an issue: GitHub.