sgl-project/sglang · error · ValueError

kitchen_w4a8 is inferred from per-layer checkpoint metadata;

Error message

kitchen_w4a8 is inferred from per-layer checkpoint metadata; it is not an online quantization method

What it means

KitchenW4A8Config.from_config always raises: like W4A4, this method is metadata-driven (per-layer markers) and has no config file representation. Calling from_config is unsupported API usage.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a8_config.py:84

    @classmethod
    def get_name(cls) -> str:
        return "kitchen_w4a8"

    @classmethod
    def get_supported_act_dtypes(cls) -> list[torch.dtype]:
        return [torch.bfloat16, torch.float16]

    @classmethod
    def get_min_capability(cls) -> int:
        return 80

    @classmethod
    def get_config_filenames(cls) -> list[str]:
        return []

    @classmethod
    def from_config(cls, config: dict[str, Any]) -> KitchenW4A8Config:
        raise ValueError(
            "kitchen_w4a8 is inferred from per-layer checkpoint metadata; "
            "it is not an online quantization method"
        )

    def get_quant_method(
        self, layer: torch.nn.Module, prefix: str
    ) -> QuantizeMethodBase | None:
        marker = self.layer_markers.get(prefix)
        if isinstance(layer, VocabParallelEmbedding):
            if marker is None:
                return None
            if marker.get("format") != "int8_tensorwise" or not marker.get(
                "_is_tensorwise_scalar"
            ):
                raise ValueError(
                    f"Unsupported quantized embedding marker for {prefix!r}: {marker}"
                )
            self.selected.append(prefix)

View on GitHub (pinned to 0132848349)

Solutions

1. Construct KitchenW4A8Config(layer_markers=...) directly from checkpoint metadata
2. Exclude kitchen_w4a8 from config-file-driven loading paths

Example fix

// before
KitchenW4A8Config.from_config(cfg)
// after
KitchenW4A8Config(layer_markers=markers)

if method_name == "kitchen_w4a8":
    cfg = KitchenW4A8Config(layer_markers=markers)  # never from_config

def supports_from_config(cls) -> bool:
    return len(cls.get_config_filenames()) > 0

When it happens

Trigger: A generic loader resolving the quant method by name and calling from_config on the class.

Common situations: Adding kitchen_w4a8 to a JSON quantization_config; registry code that assumes from_config exists for all methods.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• ComfyFp8Config must be constructed from safetensors layer ma
• comfy_nvfp4 is inferred from per-layer checkpoint metadata;
• kitchen_w4a4 is inferred from per-layer checkpoint metadata;
• Serialized W4A8 checkpoints are not supported on MPS
• Serialized W4A8 checkpoints require CUDA compute capability

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/756f56ac565f43d8. Report an issue: GitHub.