sgl-project/sglang · error · ValueError

item_first is not supported when embeddings are supplied

Error message

item_first is not supported when embeddings are supplied

What it means

Raised by TokenizerManager.score_request when item_first=True is combined with query_embed_overrides or item_embed_overrides. Embedding-override scoring constructs cross-attention inputs in a fixed query-then-item order, so the item_first reordering mode is mutually exclusive with embedding inputs.

Source

Thrown at python/sglang/srt/managers/tokenizer_manager_score_mixin.py:497

        if is_generation and label_token_ids is None:
            raise ValueError(
                "label_token_ids is required for generation (CausalLM) models."
            )
        if items is None:
            raise ValueError("items must be provided")
        if not items:
            return ScoreResult(scores=[], prompt_tokens=0)

        has_embeds = (
            query_embed_overrides is not None or item_embed_overrides is not None
        )
        if has_embeds and embed_override_token_id is None:
            raise ValueError(
                "embed_override_token_id is required when query_embed_overrides "
                "or item_embed_overrides are supplied."
            )
        if item_first and has_embeds:
            raise ValueError("item_first is not supported when embeddings are supplied")
        if item_embed_overrides is not None and len(item_embed_overrides) != len(items):
            raise ValueError(
                f"item_embed_overrides length ({len(item_embed_overrides)}) "
                f"must match items length ({len(items)})."
            )
        if self.tokenizer is not None and label_token_ids is not None:
            vocab_size = self.tokenizer.vocab_size
            for token_id in label_token_ids:
                if token_id >= vocab_size:
                    raise ValueError(
                        f"Token ID {token_id} is out of vocabulary (vocab size: {vocab_size})"
                    )

        # Check if multi-item scoring is enabled
        use_multi_item_scoring = self.server_args.enable_mis

        input_ids = None
        text_prompts = None

View on GitHub (pinned to 0132848349)

Solutions

1. Set item_first=False (or omit it) when passing query_embed_overrides/item_embed_overrides
2. If you must reorder items first, tokenize them to text prompts and drop the embed overrides
3. Pass None instead of empty lists for override arguments you don't use

Example fix

# before
await engine.async_score(queries, items, item_first=True, item_embed_overrides=embs)
# after
await engine.async_score(queries, items, item_first=False, item_embed_overrides=embs)

has_embeds = query_embed_overrides is not None or item_embed_overrides is not None
if item_first and has_embeds:
    raise ValueError("item_first cannot be combined with embedding overrides")

When it happens

Trigger: Calling engine.score(...) / async_score / score_prompts with item_first=True while also passing item_embed_overrides or query_embed_overrides (i.e., has_embeds is True).

Common situations: Porting a text-only re-ranking pipeline that used item_first=True to an embedding-based reranker API without clearing the flag; passing empty-but-non-None override lists also triggers it because has_embeds treats any non-None overrides as embeddings.

Related errors

• item_embed_overrides length ({len(item_embed_overrides)}) mu
• --speculative-draft-window-size must be positive, got {}.
• routed_dp_rank={routed_dp_rank} out of range [0, {dp_size})
• Token ID {token_id} is out of vocabulary (vocab size: {vocab
• Invalid combination of query/items types for score_request.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d9bc6c2214927645. Report an issue: GitHub.