sgl-project/sglang · critical · InternalError

Rank 0 produced no embedding for {ctx.req_id}

Error message

Rank 0 produced no embedding for {ctx.req_id}

What it means

Raised in _stage_embeddings when rank 0 has no mm_embedding to slice into per-request staged tensors. Only rank 0 stages embeddings; if the batch produced no embedding at all the staging contract is violated.

Source

Thrown at python/sglang/srt/disaggregation/encoder/server.py:1713

            )

    def _stage_embeddings(
        self,
        ctx: EncodeContext,
        requests: List[dict],
        mm_embedding: Optional[torch.Tensor],
        *,
        keep_on_gpu: bool,
    ) -> List[Tuple[int, int, int, Optional[str], Optional[int]]]:
        """Split the fused embedding per request and stage one EmbeddingData each.

        Per-request token ranges are contiguous in flatten order, so each
        staged embedding is a slice of the batch tensor.
        """
        if self.rank != 0:
            return [(0, 0, 0, None, None)] * len(requests)
        if mm_embedding is None:
            raise InternalError(f"Rank 0 produced no embedding for {ctx.req_id}")

        results = []
        staged_embeddings = []
        item_offset = 0
        token_offset = 0
        for req, num_items in zip(requests, ctx.items_per_req):
            item_end = item_offset + num_items
            num_tokens = sum(ctx.preprocess_result.token_counts[item_offset:item_end])
            embedding = mm_embedding[token_offset : token_offset + num_tokens]
            if keep_on_gpu and len(requests) > 1:
                # A view would pin the whole batch tensor until the last transfer.
                embedding = embedding.clone()
            req_aux_data = dict(ctx.aux_data)
            if ctx.aux_data.get("original_image_sizes") is not None:
                req_aux_data["original_image_sizes"] = ctx.aux_data[
                    "original_image_sizes"
                ][item_offset:item_end]
            mm_data = EmbeddingData(

View on GitHub (pinned to 0132848349)

Solutions

1. Inspect encoder logs just above for why _compute_direct_embedding yielded None
2. Confirm requests actually contain multimodal items and preprocess_result.token_counts is non-empty
3. Guard upstream: skip staging for requests with zero expected tokens
4. Report as internal bug if stock encoders hit it

if self.rank == 0 and mm_embedding is None:
    raise RuntimeError("encode returned no embedding; skip staging and fail requests")

catch InternalError in batch_encode; fail the batch explicitly instead of letting staging abort the worker

When it happens

Trigger: batch_encode reaching _stage_embeddings with mm_embedding=None on rank 0 — encode returned nothing (empty batch miscomputed, model returned None, or direct-embedding path skipped).

Common situations: Encoder model load failure that returns None instead of raising, empty items_per_req, or a custom encoder hook returning None.

Related errors

• No embedding available for Mooncake GPU-direct transfer: {re
• DSV4 HiSparse direct PD transfer currently requires the Moon
• mooncake encoder_transfer_backend requires HTTP encoders; us
• encode metadata not ready
• Encoder produced {mm_embedding.shape[0]} tokens, but preproc

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2b04136429951f0a. Report an issue: GitHub.