sgl-project/sglang · error · ValueError

item_embed_overrides length ({len(item_embed_overrides)}) mu

Error message

item_embed_overrides length ({len(item_embed_overrides)}) must match items length ({len(items)}).

What it means

Raised by score_request when item_embed_overrides is provided but its length differs from the number of items. Each item needs exactly one embedding override vector, so the lists must be the same length.

Source

Thrown at python/sglang/srt/managers/tokenizer_manager_score_mixin.py:499

                "label_token_ids is required for generation (CausalLM) models."
            )
        if items is None:
            raise ValueError("items must be provided")
        if not items:
            return ScoreResult(scores=[], prompt_tokens=0)

        has_embeds = (
            query_embed_overrides is not None or item_embed_overrides is not None
        )
        if has_embeds and embed_override_token_id is None:
            raise ValueError(
                "embed_override_token_id is required when query_embed_overrides "
                "or item_embed_overrides are supplied."
            )
        if item_first and has_embeds:
            raise ValueError("item_first is not supported when embeddings are supplied")
        if item_embed_overrides is not None and len(item_embed_overrides) != len(items):
            raise ValueError(
                f"item_embed_overrides length ({len(item_embed_overrides)}) "
                f"must match items length ({len(items)})."
            )
        if self.tokenizer is not None and label_token_ids is not None:
            vocab_size = self.tokenizer.vocab_size
            for token_id in label_token_ids:
                if token_id >= vocab_size:
                    raise ValueError(
                        f"Token ID {token_id} is out of vocabulary (vocab size: {vocab_size})"
                    )

        # Check if multi-item scoring is enabled
        use_multi_item_scoring = self.server_args.enable_mis

        input_ids = None
        text_prompts = None
        positional_embed_overrides = None
        delimiter_indices = None

View on GitHub (pinned to 0132848349)

Solutions

Regenerate item_embed_overrides so there is one embedding per element of items
If items are flattened pairs, flatten embeddings with the same order/comprehension
Log len(items) and len(item_embed_overrides) before the call to confirm alignment

Example fix

# before
item_embs = [emb_model.encode(q) for q in queries]  # wrong axis
await engine.async_score(queries, items, item_embed_overrides=item_embs)
# after
item_embs = [emb_model.encode(it) for it in items]
await engine.async_score(queries, items, item_embed_overrides=item_embs)

Defensive patterns

Strategy: validation

Validate before calling

assert item_embed_overrides is None or len(item_embed_overrides) == len(items), (
    f"{len(item_embed_overrides)=} != {len(items)=}")

Type guard

def overrides_match(items: list, embs: list | None) -> bool:
    return embs is None or len(embs) == len(items)

Prevention

Generate embeddings with the same iteration that builds items
Add a length assert in test fixtures for reranking data

When it happens

Trigger: Calling score/async_score/score_prompts with len(item_embed_overrides) != len(items), e.g. 4 items but only 3 embeddings, or embeddings batched per-query instead of per-item.

Common situations: Batched reranking where embeddings were computed per query instead of flattened per (query, item) pair; off-by-one or filtered items list without filtering the matching embeddings; reusing cached embeddings from a different item set.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/5ea9f2fa2b4b656e. Report an issue: GitHub.