{"record":{"id":"8617796df97b1e11","repo":"sgl-project/sglang","slug":"encoder-produced-mm-embedding-shape-0-tokens-b-861779","errorCode":null,"errorMessage":"Encoder produced {mm_embedding.shape[0]} tokens, but preprocessor metadata expected {expected_tokens}","messagePattern":"Encoder produced (.+?) tokens, but preprocessor metadata expected (.+?)","errorType":"http","errorClass":"InternalError","httpStatus":500,"severity":"error","filePath":"python/sglang/srt/disaggregation/encoder/server.py","lineNumber":1488,"sourceCode":"    async def _compute_embedding(\n        self,\n        ctx: EncodeContext,\n        *,\n        keep_on_gpu: bool,\n    ) -> Optional[torch.Tensor]:\n        \"\"\"Compute one flattened request with global cache as an optional stage.\"\"\"\n        if ctx.use_global_cache:\n            mm_embedding = await self._compute_global_cache_embedding(\n                ctx, keep_on_gpu=keep_on_gpu\n            )\n        else:\n            mm_embedding = await self._compute_direct_embedding(\n                ctx, keep_on_gpu=keep_on_gpu\n            )\n\n        expected_tokens = sum(ctx.preprocess_result.token_counts)\n        if mm_embedding is not None and mm_embedding.shape[0] != expected_tokens:\n            raise InternalError(\n                f\"Encoder produced {mm_embedding.shape[0]} tokens, but \"\n                f\"preprocessor metadata expected {expected_tokens}\"\n            )\n        return mm_embedding\n\n    async def _publish_preprocess_metadata(\n        self, ctx: EncodeContext, requests: List[dict]\n    ) -> None:\n        \"\"\"Publish each request's size after preprocessing, before model forward.\"\"\"\n        if self.rank != 0:\n            return\n        embedding_dim = self._embedding_dims[ctx.modality]\n        item_offset = 0\n        for request, item_count in zip(requests, ctx.items_per_req):\n            item_end = item_offset + item_count\n            token_count = sum(ctx.preprocess_result.token_counts[item_offset:item_end])\n            req_id = request[\"req_id\"]\n            state = self._require_active_encode_state(req_id)","sourceCodeStart":1470,"sourceCodeEnd":1506,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/disaggregation/encoder/server.py#L1470-L1506","documentation":"Raised by the disaggregated multimodal encoder server when the encoder backend returns an embedding tensor whose token count (rows) disagrees with the token counts recorded by the preprocessor for the same request. It is a sanity check in _compute_embedding that guards the contract between preprocessing metadata and the actual model output.","triggerScenarios":"Calling batch_encode (which awaits _compute_direct_embedding) where sum(ctx.preprocess_result.token_counts) != mm_embedding.shape[0]; e.g. a vision model whose patch/token math changed, or a custom encoder returning padded/truncated embeddings.","commonSituations":"Upgrading a multimodal model whose processor now emits different token counts, mismatched processor/model revisions, or a custom _compute_direct_embedding implementation that reshapes embeddings incorrectly.","solutions":["Verify the preprocessor version matches the model checkpoint (same processor config)","Log ctx.preprocess_result.token_counts and mm_embedding.shape side by side for the failing request to find which item diverges","If a custom encoder is registered, audit its reshape/concatenation logic to emit exactly expected_tokens rows","Report upstream if stock models trigger it — indicates encoder/preprocessor contract bug"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"expected = sum(ctx.preprocess_result.token_counts)\nassert mm_embedding is None or mm_embedding.shape[0] == expected, (mm_embedding.shape[0], expected)","typeGuard":null,"tryCatchPattern":"catch sglang InternalError around batch_encode; log token_counts vs shape and fail the single request, not the server","preventionTips":["Pin processor and model to the same revision","Add a unit test asserting token counts for sample multimodal inputs"],"tags":["disaggregation","multimodal","encoder","shape-mismatch"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}