sgl-project/sglang · critical · ValueError

Unsupported quantized linear marker for {prefix!r}

Error message

Unsupported quantized linear marker for {prefix!r}

What it means

Raised by ComfyNvfp4Config.get_quant_method when a LinearBase layer has a quantization marker whose format is not 'nvfp4'. This config dispatches only nvfp4-serialized linear layers; markers with other formats (or unquantized layers mislabeled in the checkpoint) fail fast during weight loading.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/comfy_nvfp4.py:237

    ) -> QuantizeMethodBase | None:
        marker = self.layer_markers.get(prefix)
        if isinstance(layer, VocabParallelEmbedding):
            if marker is None:
                return None
            if marker.get("format") != "int8_tensorwise" or not marker.get(
                "_is_rowwise"
            ):
                raise ValueError(
                    f"Unsupported quantized embedding marker for {prefix!r}: {marker}"
                )
            self.selected.append(prefix)
            return ComfyRowwiseInt8EmbeddingMethod()
        if not isinstance(layer, LinearBase):
            return None
        if marker is None:
            return UnquantizedLinearMethod()
        if marker.get("format") != "nvfp4":
            raise ValueError(f"Unsupported quantized linear marker for {prefix!r}")
        self.selected.append(prefix)
        return ComfyFullPrecisionNvfp4LinearMethod(
            self,
            has_pre_quant_scale=bool(marker.get("_has_pre_quant_scale")),
        )

    def quantizes_embedding(self, prefix: str) -> bool:
        marker = self.layer_markers.get(prefix)
        return bool(
            marker is not None
            and marker.get("format") == "int8_tensorwise"
            and marker.get("_is_rowwise")
        )


__all__ = [
    "ComfyFullPrecisionNvfp4LinearMethod",
    "ComfyNvfp4Config",

View on GitHub (pinned to 0132848349)

Solutions

1. Re-quantize/export the model so all linear markers have format 'nvfp4'
2. Strip the offending marker so marker is None and the layer falls back to UnquantizedLinearMethod
3. Route layers with other formats through a config that supports them (e.g. kitchen_int8)

Example fix

// before
{"format": "int8_tensorwise"}
// after
{"format": "nvfp4", "_has_pre_quant_scale": false}

for prefix, m in layer_markers.items():
    if is_linear_prefix(prefix) and m is not None:
        assert m.get("format") == "nvfp4", (prefix, m)

def is_nvfp4_marker(m: dict) -> bool:
    return m.get("format") == "nvfp4"

When it happens

Trigger: Loading a checkpoint where a linear layer's marker format is e.g. 'int8_tensorwise' or 'awq' while the model runs under the comfy_nvfp4 quantization config.

Common situations: Mixing quantization schemes in one ComfyUI export; stale layer_markers dict from a previous quantization run; renaming formats between exporter versions.

Related errors

• Unsupported quantized embedding marker for {prefix!r}: {mark
• SGLang diffusion currently supports AutoRound auto_gptq chec
• AutoRound fused module {target!r} has inconsistent shard con
• Parameter {param_name} not found in the model.
• Unsupported Comfy INT8 format for {prefix!r}: {marker.get('f

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7d0f72124a503d35. Report an issue: GitHub.