sgl-project/sglang · error · NotImplementedError

f"The class {type(quant_method).__name__} must implement the

Error message

f"The class {type(quant_method).__name__} must implement the 'embedding' method, see UnquantizedEmbeddingMethod."

What it means

When constructing a VocabParallelEmbedding, sglang verifies the provided quant_method class implements the 'embedding' forward method (as UnquantizedEmbeddingMethod does). If the quantization method class lacks an embedding() implementation, this NotImplementedError is raised, since embedding layers cannot use the generic linear-only quant method.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/vocab_parallel_embedding.py:273

            self.tp_size,
        )
        self.embedding_dim = embedding_dim

        quant_method = None
        if quant_config is not None:
            quant_method = quant_config.get_quant_method(self, prefix=prefix)
        if quant_method is None:
            quant_method = UnquantizedEmbeddingMethod()

        # If we are making an embedding layer, then our quantization linear
        # method must implement the embedding operation. If we are another
        # layer type like ParallelLMHead, this is not important.
        is_embedding_layer = type(self.__class__) is VocabParallelEmbedding
        quant_method_implements_embedding = method_has_implemented_embedding(
            type(quant_method)
        )
        if is_embedding_layer and not quant_method_implements_embedding:
            raise NotImplementedError(
                f"The class {type(quant_method).__name__} must implement "
                "the 'embedding' method, see UnquantizedEmbeddingMethod."
            )

        self.quant_method: QuantizeMethodBase = quant_method

        if params_dtype is None:
            params_dtype = torch.get_default_dtype()
        # Divide the weight matrix along the vocaburaly dimension.
        self.num_added_embeddings = self.num_embeddings - self.org_vocab_size
        self.num_embeddings_per_partition = divide(
            self.num_embeddings_padded, self.tp_size
        )
        assert (
            self.shard_indices.num_elements_padded == self.num_embeddings_per_partition
        )
        self.num_org_embeddings_per_partition = (
            self.shard_indices.org_vocab_end_index

View on GitHub (pinned to 0132848349)

Solutions

1. Implement an embedding() method on the quant method class, mirroring UnquantizedEmbeddingMethod
2. If the quant method is for linear layers only, use UnquantizedEmbeddingMethod for the embedding layer instead
3. When subclassing, ensure method_has_implemented_embedding can see the override — define embedding() directly, not via __getattr__

Example fix

# before
class MyQuantMethod(QuantizeMethodBase):
    def apply(self, layer, x, bias): ...
# after
class MyQuantMethod(QuantizeMethodBase):
    def apply(self, layer, x, bias): ...
    def embedding(self, layer, x): return F.embedding(x, layer.weight)

from sglang.multimodal_gen.runtime.layers.vocab_parallel_embedding import method_has_implemented_embedding

assert method_has_implemented_embedding(type(quant_method)), "quant method lacks embedding()"

def supports_embedding(qm) -> bool:
    return callable(getattr(qm, "embedding", None))

When it happens

Trigger: Passing a QuantizeMethodBase subclass that only implements apply() (a linear-only quantizer) as quant_method to VocabParallelEmbedding; wiring a new/custom quant method into an embedding layer without adding an embedding() method.

Common situations: Adding a new quantization format to sglang and reusing an existing linear quant method for embeddings; subclassing a quant method and overriding only apply(); version mismatches where an older quant class lacks embedding support.

Related errors

• Comfy INT8 embedding weights support lookup only
• QVGPackedCausalKVCache does not support pinned-sink (longliv
• bitsandbytes 4-bit TP only supports column-parallel output s
• bitsandbytes 4-bit TP does not support nested quant states.
• Unsupported quantized embedding marker for {prefix!r}: {mark

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7df99e3e3525cf78. Report an issue: GitHub.