{"record":{"id":"2b04136429951f0a","repo":"sgl-project/sglang","slug":"rank-0-produced-no-embedding-for-ctx-req-id","errorCode":null,"errorMessage":"Rank 0 produced no embedding for {ctx.req_id}","messagePattern":"Rank 0 produced no embedding for (.+?)","errorType":"http","errorClass":"InternalError","httpStatus":500,"severity":"critical","filePath":"python/sglang/srt/disaggregation/encoder/server.py","lineNumber":1713,"sourceCode":"            )\n\n    def _stage_embeddings(\n        self,\n        ctx: EncodeContext,\n        requests: List[dict],\n        mm_embedding: Optional[torch.Tensor],\n        *,\n        keep_on_gpu: bool,\n    ) -> List[Tuple[int, int, int, Optional[str], Optional[int]]]:\n        \"\"\"Split the fused embedding per request and stage one EmbeddingData each.\n\n        Per-request token ranges are contiguous in flatten order, so each\n        staged embedding is a slice of the batch tensor.\n        \"\"\"\n        if self.rank != 0:\n            return [(0, 0, 0, None, None)] * len(requests)\n        if mm_embedding is None:\n            raise InternalError(f\"Rank 0 produced no embedding for {ctx.req_id}\")\n\n        results = []\n        staged_embeddings = []\n        item_offset = 0\n        token_offset = 0\n        for req, num_items in zip(requests, ctx.items_per_req):\n            item_end = item_offset + num_items\n            num_tokens = sum(ctx.preprocess_result.token_counts[item_offset:item_end])\n            embedding = mm_embedding[token_offset : token_offset + num_tokens]\n            if keep_on_gpu and len(requests) > 1:\n                # A view would pin the whole batch tensor until the last transfer.\n                embedding = embedding.clone()\n            req_aux_data = dict(ctx.aux_data)\n            if ctx.aux_data.get(\"original_image_sizes\") is not None:\n                req_aux_data[\"original_image_sizes\"] = ctx.aux_data[\n                    \"original_image_sizes\"\n                ][item_offset:item_end]\n            mm_data = EmbeddingData(","sourceCodeStart":1695,"sourceCodeEnd":1731,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/disaggregation/encoder/server.py#L1695-L1731","documentation":"Raised in _stage_embeddings when rank 0 has no mm_embedding to slice into per-request staged tensors. Only rank 0 stages embeddings; if the batch produced no embedding at all the staging contract is violated.","triggerScenarios":"batch_encode reaching _stage_embeddings with mm_embedding=None on rank 0 — encode returned nothing (empty batch miscomputed, model returned None, or direct-embedding path skipped).","commonSituations":"Encoder model load failure that returns None instead of raising, empty items_per_req, or a custom encoder hook returning None.","solutions":["Inspect encoder logs just above for why _compute_direct_embedding yielded None","Confirm requests actually contain multimodal items and preprocess_result.token_counts is non-empty","Guard upstream: skip staging for requests with zero expected tokens","Report as internal bug if stock encoders hit it"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"if self.rank == 0 and mm_embedding is None:\n    raise RuntimeError(\"encode returned no embedding; skip staging and fail requests\")","typeGuard":null,"tryCatchPattern":"catch InternalError in batch_encode; fail the batch explicitly instead of letting staging abort the worker","preventionTips":["Assert encode results non-None in test harness (see existing test_mooncake_embedding_is_ready_only_after_cuda_sync)","Guard empty multimodal items before encode"],"tags":["encoder","disaggregation","mooncake","staging"],"backgroundTag":"missing-required-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}