{"record":{"id":"802ff71023d5d4ac","repo":"sgl-project/sglang","slug":"encoder-produced-mm-embedding-shape-0-tokens-b","errorCode":null,"errorMessage":"Encoder produced {mm_embedding.shape[0]} tokens, but preprocessor metadata expected {offset}","messagePattern":"Encoder produced (.+?) tokens, but preprocessor metadata expected (.+?)","errorType":"http","errorClass":"InternalError","httpStatus":500,"severity":"error","filePath":"python/sglang/srt/disaggregation/encoder/server.py","lineNumber":784,"sourceCode":"                if val and int(val) > 0:\n                    dims[Modality.AUDIO] = int(val)\n                    break\n\n        logger.info(f\"Global cache embedding dims: {dims}\")\n        return dims\n\n    def slice_embedding(\n        self,\n        mm_embedding: torch.Tensor,\n        token_counts: Iterable[int],\n    ) -> List[torch.Tensor]:\n        \"\"\"Slice embeddings using preprocessing-owned token counts.\"\"\"\n        slices, offset = [], 0\n        for count in token_counts:\n            slices.append(mm_embedding[offset : offset + count])\n            offset += count\n        if mm_embedding.shape[0] != offset:\n            raise InternalError(\n                f\"Encoder produced {mm_embedding.shape[0]} tokens, but \"\n                f\"preprocessor metadata expected {offset}\"\n            )\n        return slices\n\n    def _calculate_hashes_from_features(\n        self, mm_feature, grid_thw: List, modality: Modality, mm_inputs=None\n    ) -> List[int]:\n        \"\"\"CPU Task: Compute hashes based on processed feature patches.\"\"\"\n        preprocessed_items = (\n            get_encoder_preprocessed_items(mm_inputs) if mm_inputs is not None else None\n        )\n        if preprocessed_items is not None:\n            if len(preprocessed_items) != len(grid_thw):\n                raise ValueError(\n                    \"Encoder preprocess item/grid mismatch: \"\n                    f\"{len(preprocessed_items)} items != {len(grid_thw)} grids\"\n                )","sourceCodeStart":766,"sourceCodeEnd":802,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/disaggregation/encoder/server.py#L766-L802","documentation":"slice_embedding splits the encoder output into per-item slices using preprocessing-owned token_counts and validates that the total offset equals the encoder's produced token count. A mismatch means the encoder emitted a different number of multimodal tokens than the preprocessor predicted, indicating a serious encoder/preprocessor disagreement.","triggerScenarios":"Calling slice_embedding (via _encode_missing) where the vision encoder produced more or fewer tokens than the preprocessor's metadata predicted — e.g. different image resolution handling, patch/grid config mismatch (as in the Kimi-VL 2D grid case), or different processor versions on each side.","commonSituations":"Encoder and decoder/preprocessor using different versions of the multimodal processor or different image processing configs; a model integration where grid counting logic diverges from the encoder's actual behavior; resizing/cropping settings inconsistent between components.","solutions":["Verify encoder and preprocessor run the same model processor version and config (image resolutions, patch size, grid settings)","Reproduce with the failing image and compare token_counts from the preprocessor against the encoder's output shape","Fix the preprocessor's token counting (e.g. 2D grid logic) to match encoder behavior, or update the encoder config","Pin consistent package versions across encoder and decoder deployments"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"total = sum(token_counts)\nassert mm_embedding.shape[0] == total, f'token drift: {mm_embedding.shape[0]} vs {total}'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Pin identical processor versions on encoder and preprocessor","Add per-image regression tests comparing predicted vs actual token counts","Gate model integrations on a slice_embedding consistency test"],"tags":["multimodal","token-count","encoder","preprocessor","mismatch"],"backgroundTag":"token-count-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}