sgl-project/sglang · critical · ValueError

Serialized W4A4 layer {prefix!r} has unsupported linear_dtyp

Error message

Serialized W4A4 layer {prefix!r} has unsupported linear_dtype={linear_dtype!r}; expected one of {_SUPPORTED_LINEAR_DTYPES}

What it means

_parse_marker validates linear_dtype (default 'int4') against _SUPPORTED_LINEAR_DTYPES. Only specific 4-bit storage dtypes have W4A4 kernels, so an unknown dtype string (e.g. 'int8', 'fp4_e2m1') is rejected.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a4_config.py:133

            )
        self.selected.append(prefix)
        return KitchenW4A4LinearMethod(
            convrot_group_size=convrot_group_size,
            linear_dtype=linear_dtype,
        )

    @staticmethod
    def _parse_marker(prefix: str, marker: dict[str, Any]) -> tuple[int, str]:
        convrot_group_size = int(marker.get("convrot_groupsize", 256))
        if convrot_group_size not in _SUPPORTED_CONVROT_GROUP_SIZES:
            raise ValueError(
                f"Serialized W4A4 layer {prefix!r} has unsupported "
                f"convrot_groupsize={convrot_group_size}; expected one of "
                f"{_SUPPORTED_CONVROT_GROUP_SIZES}"
            )
        linear_dtype = str(marker.get("linear_dtype", "int4"))
        if linear_dtype not in _SUPPORTED_LINEAR_DTYPES:
            raise ValueError(
                f"Serialized W4A4 layer {prefix!r} has unsupported "
                f"linear_dtype={linear_dtype!r}; expected one of "
                f"{_SUPPORTED_LINEAR_DTYPES}"
            )
        return convrot_group_size, linear_dtype

    @staticmethod
    def _supports_input_size(input_size: int, convrot_group_size: int) -> bool:
        return (
            input_size % _QUANT_GROUP_SIZE == 0 and input_size % convrot_group_size == 0
        )

    def supports_input_partition(
        self, prefix: str, input_size_per_partition: int
    ) -> bool:
        marker = self.layer_markers.get(prefix)
        if marker is None:
            return True

View on GitHub (pinned to 0132848349)

Solutions

1. Set linear_dtype to a supported value (check _SUPPORTED_LINEAR_DTYPES, e.g. 'int4')
2. Align exporter and sglang runtime versions

Example fix

// before
{"linear_dtype": "nf4"}
// after
{"linear_dtype": "int4"}

from ...kitchen_w4a4_config import _SUPPORTED_LINEAR_DTYPES
assert marker.get("linear_dtype", "int4") in _SUPPORTED_LINEAR_DTYPES

def valid_w4a4_linear_dtype(m: dict) -> bool:
    return str(m.get("linear_dtype", "int4")) in _SUPPORTED_LINEAR_DTYPES

When it happens

Trigger: A marker with "linear_dtype": "fp4" or another string not in _SUPPORTED_LINEAR_DTYPES.

Common situations: Exporter emitting new dtype names the installed runtime doesn't know; typos in hand-written markers.

Related errors

• {name} must be float32, got {scale.dtype}
• Validate failed: unsupported dtype: {t.dtype}
• kv-canary: {name} must have dtype {dtype}, got {tensor.dtype
• num_token_non_padded must be an integer tensor, got {num_tok
• NPU packed attention requires q, k, and v with the same dtyp

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b36b06d5198e0ff5. Report an issue: GitHub.