sgl-project/sglang · error · NotImplementedError

Unsupported Comfy NVFP4 companion format(s): + ", ".join(sor

Error message

Unsupported Comfy NVFP4 companion format(s): + ", ".join(sorted(unsupported))

What it means

When a MiniMax-H3 checkpoint's layer markers (from quantization metadata) include the 'nvfp4' format, the resolver only accepts 'nvfp4', 'int8_tensorwise', and 'float8_e4m3fn' as companion per-layer formats. Any other format appearing alongside NVFP4 is not implemented and raises NotImplementedError.

Source

Thrown at python/sglang/multimodal_gen/runtime/loader/minimax_h3_weights.py:62

                        "MiniMax-H3 checkpoint shards disagree on adaln_t_table "
                        f"shape: {adaln_curve_shape} vs {shape}"
                    )
                adaln_curve_shape = shape

    return adaln_curve_shape, layer_markers


def resolve_minimax_h3_checkpoint_quantization(
    layer_markers: dict[str, dict[str, Any]],
    safetensors_list: list[str] | None = None,
    param_names_mapping: dict | None = None,
    reverse_param_names_mapping: dict | None = None,
) -> QuantizationConfig | None:
    formats = {str(marker.get("format")) for marker in layer_markers.values()}
    if "nvfp4" in formats:
        unsupported = formats - {"nvfp4", "int8_tensorwise", "float8_e4m3fn"}
        if unsupported:
            raise NotImplementedError(
                "Unsupported Comfy NVFP4 companion format(s): "
                + ", ".join(sorted(unsupported))
            )
        if safetensors_list is None:
            raise ValueError("MiniMax-H3 NVFP4 metadata requires checkpoint files")
        config = build_nvfp4_config_from_safetensors_list(
            safetensors_list,
            param_names_mapping,
            reverse_param_names_mapping,
        )
        if not isinstance(config, ModelOptFp4Config):
            raise ValueError("Could not resolve MiniMax-H3 NVFP4 checkpoint layout")
        config.set_comfy_layer_markers(layer_markers)
        config.checkpoint_uses_comfy_quantization = True
        config.checkpoint_uses_native_qkv_layout = True
        config.checkpoint_weight_scale_layout = "swizzled"
        config.swap_weight_nibbles = True
        return config

View on GitHub (pinned to 0132848349)

Solutions

1. Re-quantize so companion layers use only int8_tensorwise or float8_e4m3fn alongside nvfp4
2. Check the marker strings in the checkpoint metadata to find which layers carry the unsupported format and leave those unquantized or int8
3. Use a prebuilt NVFP4 checkpoint distribution known to be compatible

Example fix

# before: markers = {'layers.0': {'format': 'nvfp4'}, 'layers.3': {'format': 'q4k'}} -> NotImplementedError
# after: markers['layers.3'] = {'format': 'int8_tensorwise'}

ALLOWED = {'nvfp4', 'int8_tensorwise', 'float8_e4m3fn'}
formats = {str(m.get('format')) for m in layer_markers.values()}
if 'nvfp4' in formats:
    bad = formats - ALLOWED
    assert not bad, f'unsupported companion formats: {sorted(bad)}'

def nvfp4_markers_supported(layer_markers: dict) -> bool:
    formats = {str(m.get('format')) for m in layer_markers.values()}
    return 'nvfp4' not in formats or formats <= {'nvfp4', 'int8_tensorwise', 'float8_e4m3fn'}

try:
    q = resolve_minimax_h3_checkpoint_quantization(markers, files, mapping, rev)
except NotImplementedError as e:
    if 'NVFP4 companion' in str(e):
        raise UnsupportedQuantMix(sorted({str(m.get('format')) for m in markers.values()})) from e
    raise

When it happens

Trigger: Calling resolve_minimax_h3_checkpoint_quantization (via load_customized) on a checkpoint whose serialized layer markers mix 'nvfp4' with formats like 'int4', 'awq', 'fp8_dynamic', or unknown strings.

Common situations: ComfyUI NVFP4 exports that also quantize some layers with a different scheme; checkpoints converted by third-party tools that emit nonstandard format marker strings; partially requantized NVFP4 checkpoints.

Related errors

• Type must match: {self.a_dtype} != {self.b_dtype}
• nvfp4_gemm_swiglu_nvfp4_quant currently supports NVFP4 input
• Shape mismatch: A K={k}, B K={b.shape[1] * 2}
• Output N={n_out} must be divisible by sf_vec_size={sf_vec_si
• Comfy full_precision_matrix_mult does not support fused line

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c8e5deb4f850130d. Report an issue: GitHub.