sgl-project/sglang · error · ValueError

kitchen_w4a4 is inferred from per-layer checkpoint metadata;

Error message

kitchen_w4a4 is inferred from per-layer checkpoint metadata; it is not an online quantization method

What it means

KitchenW4A4Config.from_config always raises: the config has no serialized config file (get_config_filenames returns []) and is constructed purely from per-layer checkpoint metadata. Calling from_config is an API misuse.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a4_config.py:90

    @classmethod
    def get_name(cls) -> str:
        return "kitchen_w4a4"

    @classmethod
    def get_supported_act_dtypes(cls) -> list[torch.dtype]:
        return [torch.bfloat16, torch.float16]

    @classmethod
    def get_min_capability(cls) -> int:
        return 75

    @classmethod
    def get_config_filenames(cls) -> list[str]:
        return []

    @classmethod
    def from_config(cls, config: dict[str, Any]) -> KitchenW4A4Config:
        raise ValueError(
            "kitchen_w4a4 is inferred from per-layer checkpoint metadata; "
            "it is not an online quantization method"
        )

    def get_quant_method(
        self, layer: torch.nn.Module, prefix: str
    ) -> QuantizeMethodBase | None:
        if not isinstance(layer, LinearBase):
            return None
        marker = self.layer_markers.get(prefix)
        if marker is None:
            return UnquantizedLinearMethod()
        if marker.get("format") == "int8_tensorwise":
            assert self._int8_config is not None
            method = self._int8_config.get_quant_method(layer, prefix)
            self.selected.append(prefix)
            return method

View on GitHub (pinned to 0132848349)

Solutions

1. Construct KitchenW4A4Config directly with layer_markers extracted from the checkpoint
2. Do not list kitchen_w4a4 in config-file-driven quantization maps

Example fix

// before
KitchenW4A4Config.from_config({})
// after
KitchenW4A4Config(layer_markers=extract_layer_markers(checkpoint))

if method_name == "kitchen_w4a4":
    cfg = KitchenW4A4Config(layer_markers=markers)  # never from_config

def supports_from_config(cls) -> bool:
    return len(cls.get_config_filenames()) > 0

When it happens

Trigger: Any code path that reads a quant config JSON and calls KitchenW4A4Config.from_config(config), e.g. a generic loader keyed by quant method name.

Common situations: Generic quantization-registry code assuming every method implements from_config; adding 'kitchen_w4a4' to a hf quantization_config map.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• The quantization method `{quantization}` is already exists.
• The quantization config must be a subclass of `QuantizationC
• Invalid quantization method: {quantization}
• ComfyFp8Config must be constructed from safetensors layer ma
• comfy_nvfp4 is inferred from per-layer checkpoint metadata;

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/44ec3833e97d8f08. Report an issue: GitHub.