sgl-project/sglang · error · ValueError

embed_override_token_id is required when query_embed_overrid

Error message

embed_override_token_id is required when query_embed_overrides or item_embed_overrides are supplied.

What it means

When query_embed_overrides or item_embed_overrides are supplied, embed_override_token_id must also be given so the server knows which token marks each override slot. The has_embeds check rejects embedding overrides without a placeholder token id.

Source

Thrown at python/sglang/srt/managers/tokenizer_manager_score_mixin.py:492

        return_pooled_hidden_states is only supported for non-generation models
        (SequenceClassification, RewardModel); raises ValueError for CausalLM.
        """
        is_generation = self.is_generation

        if is_generation and label_token_ids is None:
            raise ValueError(
                "label_token_ids is required for generation (CausalLM) models."
            )
        if items is None:
            raise ValueError("items must be provided")
        if not items:
            return ScoreResult(scores=[], prompt_tokens=0)

        has_embeds = (
            query_embed_overrides is not None or item_embed_overrides is not None
        )
        if has_embeds and embed_override_token_id is None:
            raise ValueError(
                "embed_override_token_id is required when query_embed_overrides "
                "or item_embed_overrides are supplied."
            )
        if item_first and has_embeds:
            raise ValueError("item_first is not supported when embeddings are supplied")
        if item_embed_overrides is not None and len(item_embed_overrides) != len(items):
            raise ValueError(
                f"item_embed_overrides length ({len(item_embed_overrides)}) "
                f"must match items length ({len(items)})."
            )
        if self.tokenizer is not None and label_token_ids is not None:
            vocab_size = self.tokenizer.vocab_size
            for token_id in label_token_ids:
                if token_id >= vocab_size:
                    raise ValueError(
                        f"Token ID {token_id} is out of vocabulary (vocab size: {vocab_size})"
                    )

View on GitHub (pinned to 0132848349)

Solutions

1. Pass embed_override_token_id (the token id you inserted into query/items text as placeholder) alongside the overrides
2. If embeddings weren't intended, remove the *_embed_overrides arguments

Example fix

# before
result = engine.score_request(query=..., items=...,
    query_embed_overrides=embs)

# after
result = engine.score_request(query=..., items=...,
    query_embed_overrides=embs, embed_override_token_id=PLACEHOLDER_ID)

has_embeds = (query_embed_overrides is not None or item_embed_overrides is not None)
if has_embeds:
    assert embed_override_token_id is not None

def embed_kwargs_consistent(kwargs: dict) -> bool:
    has = kwargs.get("query_embed_overrides") is not None or kwargs.get("item_embed_overrides") is not None
    return (not has) or kwargs.get("embed_override_token_id") is not None

When it happens

Trigger: Calling score/score_request with query_embed_overrides (and/or item_embed_overrides) while omitting embed_override_token_id.

Common situations: Multimodal pipelines where embeddings are passed but the placeholder token constant isn't plumbed through a config; API wrappers with the token id optional and defaulted to None.

Understand the failure class

Background: Missing required parameter errors: what 'X is required' and 'the required X param is missing' mean, and how to fix them — this error's family across 27 libraries.

Related errors

• {label} contains {len(positions)} occurrences of embed_overr
• Value error, parameter top_n should be larger than 0.
• label_token_ids is required for generation (CausalLM) models
• items must be provided
• v_cache must be provided

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/871bf0a004f10717. Report an issue: GitHub.