sgl-project/sglang · critical · ValueError

Unsupported quantized embedding marker for {prefix!r}: {mark

Error message

Unsupported quantized embedding marker for {prefix!r}: {marker}

What it means

Raised by ComfyNvfp4Config.get_quant_method when a VocabParallelEmbedding layer has a checkpoint quantization marker whose format is not 'int8_tensorwise' with _is_rowwise=true. The Comfy nvfp4 quant config only supports rowwise int8-quantized embeddings; any other marker format on an embedding layer is rejected at model-load time.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/comfy_nvfp4.py:227

    @classmethod
    def from_config(cls, config: dict[str, Any]) -> ComfyNvfp4Config:
        raise ValueError(
            "comfy_nvfp4 is inferred from per-layer checkpoint metadata; "
            "it is not an online quantization method"
        )

    def get_quant_method(
        self, layer: nn.Module, prefix: str
    ) -> QuantizeMethodBase | None:
        marker = self.layer_markers.get(prefix)
        if isinstance(layer, VocabParallelEmbedding):
            if marker is None:
                return None
            if marker.get("format") != "int8_tensorwise" or not marker.get(
                "_is_rowwise"
            ):
                raise ValueError(
                    f"Unsupported quantized embedding marker for {prefix!r}: {marker}"
                )
            self.selected.append(prefix)
            return ComfyRowwiseInt8EmbeddingMethod()
        if not isinstance(layer, LinearBase):
            return None
        if marker is None:
            return UnquantizedLinearMethod()
        if marker.get("format") != "nvfp4":
            raise ValueError(f"Unsupported quantized linear marker for {prefix!r}")
        self.selected.append(prefix)
        return ComfyFullPrecisionNvfp4LinearMethod(
            self,
            has_pre_quant_scale=bool(marker.get("_has_pre_quant_scale")),
        )

    def quantizes_embedding(self, prefix: str) -> bool:
        marker = self.layer_markers.get(prefix)

View on GitHub (pinned to 0132848349)

Solutions

1. Re-export/quantize the model so embedding markers use format 'int8_tensorwise' with _is_rowwise: true
2. Remove the embedding entry from layer_markers so marker is None and the embedding stays unquantized
3. Move the model to a quantization config that supports the embedding format actually present

Example fix

// before
layer_markers = {"model.embed_tokens": {"format": "nvfp4"}}
// after
layer_markers = {"model.embed_tokens": {"format": "int8_tensorwise", "_is_rowwise": True}}

marker = layer_markers.get(prefix)
if isinstance(layer, VocabParallelEmbedding) and marker is not None:
    assert marker.get("format") == "int8_tensorwise" and marker.get("_is_rowwise"), marker

def is_comfy_rowwise_int8_marker(m: dict) -> bool:
    return m.get("format") == "int8_tensorwise" and bool(m.get("_is_rowwise"))

When it happens

Trigger: Loading a ComfyUI-exported checkpoint whose embedding layer marker has format != 'int8_tensorwise' (e.g. 'nvfp4' or 'fp8') or is missing _is_rowwise, while the model config selects the comfy_nvfp4 quantization method.

Common situations: Exporting a model from ComfyUI with mixed quantization recipes (embedding quantized with a different scheme than linear layers), or hand-editing layer_markers in the checkpoint config.

Related errors

• Unsupported quantized linear marker for {prefix!r}
• Unsupported quantized embedding marker for {prefix!r}: {mark
• SGLang diffusion currently supports AutoRound auto_gptq chec
• AutoRound fused module {target!r} has inconsistent shard con
• Parameter {param_name} not found in the model.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/848a863275a7f54b. Report an issue: GitHub.