sgl-project/sglang · error · ValueError

embed_override_token_id is required when embed_overrides is

Error message

embed_override_token_id is required when embed_overrides is provided

What it means

The embeddings endpoint enforces pairing between request.embed_overrides (per-token embedding overrides) and request.embed_override_token_id (the token id the overrides attach to). Providing overrides without a token id is rejected in _convert_to_internal_request.

Source

Thrown at python/sglang/srt/entrypoints/openai/serving_embedding.py:158

                        "image_data": images,
                        "video_data": videos,
                    }
            else:
                # List of integers (token IDs) or empty list
                prompt_kwargs = {"input_ids": prompt}
        else:
            # Other types (should not happen but handle gracefully)
            prompt_kwargs = {"input_ids": prompt}

        # Resolve LoRA adapter from model parameter or explicit lora_path
        lora_path = self._resolve_lora_path(request.model, request.lora_path)

        # Validate pairing: both or neither must be provided
        if (
            request.embed_overrides is not None
            and request.embed_override_token_id is None
        ):
            raise ValueError(
                "embed_override_token_id is required when embed_overrides is provided"
            )
        if (
            request.embed_override_token_id is not None
            and request.embed_overrides is None
        ):
            raise ValueError(
                "embed_override_token_id requires embed_overrides to be provided"
            )

        # Convert float lists to tensors; position resolution is deferred
        # to the tokenizer manager (after tokenization for text inputs).
        embed_overrides = convert_embeds_to_tensors(request.embed_overrides)

        adapted_request = EmbeddingReqInput(
            **prompt_kwargs,
            rid=request.rid,
            priority=request.priority,

View on GitHub (pinned to 0132848349)

Solutions

1. Set embed_override_token_id alongside embed_overrides
2. Omit both fields if you don't need overrides

Example fix

# before
EmbeddingReqInput(input="x", embed_overrides=[0.1, 0.2])
# after
EmbeddingReqInput(input="x", embed_overrides=[0.1, 0.2], embed_override_token_id=128000)

if req.embed_overrides is not None: assert req.embed_override_token_id is not None

def overrides_paired(req) -> bool:
    return (req.embed_overrides is None) == (req.embed_override_token_id is None)

When it happens

Trigger: POST /v1/embeddings with embed_overrides set (non-None) but embed_override_token_id omitted/None.

Common situations: Building embedding-override requests programmatically and forgetting the token id after refactors; partial copies of example payloads.

Understand the failure class

Background: Missing required parameter errors: what 'X is required' and 'the required X param is missing' mean, and how to fix them — this error's family across 27 libraries.

Related errors

• embed_override_token_id requires embed_overrides to be provi
• cache_dit_params must be a dict, got {type(raw).__name__}.
• Unknown cache_dit_params keys: {sorted(unknown)}. Valid keys
• cache_dit_params['secondary'] must be a dict, got {type(seco
• Unknown cache_dit_params['secondary'] keys: {sorted(unknown)

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2d5500c8e5cbbbed. Report an issue: GitHub.