{"record":{"id":"5064476f544104b9","repo":"sgl-project/sglang","slug":"kimi-k3-expects-one-vision-grid-per-multimodaldata","errorCode":null,"errorMessage":"Kimi-K3 expects one vision grid per MultimodalDataItem; split aggregated encoder inputs before get_image_feature()","messagePattern":"Kimi-K3 expects one vision grid per MultimodalDataItem; split aggregated encoder inputs before get_image_feature\\(\\)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/kimi_k3.py","lineNumber":3368,"sourceCode":"            mm_data,\n            image_processor,\n            use_gpu_preprocessing=use_gpu_preprocessing,\n        )\n\n    def get_image_feature(self, items: List[MultimodalDataItem]) -> torch.Tensor:\n        device = self.vision_tower.device\n        target_dtype = self.vision_tower.patch_embed.proj.weight.dtype\n        image_grid_thws = []\n        for item in items:\n            grid_thw = item.model_specific_data.get(\"image_grid_thw\")\n            if grid_thw is None:\n                grid_thw = item.model_specific_data[\"grid_thws\"]\n            if grid_thw.shape[0] != 1:\n                # One item must carry exactly one logical image so the DP\n                # owner assignment and the bounded CUDA-IPC lease accounting\n                # stay per-item; aggregated encoder inputs are split upstream\n                # (EPD encode server) before reaching this point.\n                raise ValueError(\n                    \"Kimi-K3 expects one vision grid per MultimodalDataItem; \"\n                    \"split aggregated encoder inputs before get_image_feature()\"\n                )\n            image_grid_thws.append(grid_thw)\n        grid_thws_host = torch.concat(image_grid_thws, dim=0).cpu()\n        grid_thw_list = grid_thws_host.tolist()\n\n        def materialize_item_features(image_indices: List[int]) -> torch.Tensor:\n            \"\"\"Materialize only the images assigned to this vision-DP rank.\"\"\"\n            from sglang.srt.multimodal.encoder_preprocessing import (\n                LOCAL_PREPROCESSED_KEY,\n            )\n\n            # Match the configured TP consumer count captured when the\n            # tokenizer creates MmItemMemoryPool. A live attention subgroup\n            # size could leave acknowledgements missing and strand the lease.\n            ipc_consumer_count = max(get_parallel().tp_size, 1)\n            device_index = device.index","sourceCodeStart":3350,"sourceCodeEnd":3386,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/kimi_k3.py#L3350-L3386","documentation":"get_image_feature() requires each MultimodalDataItem to carry exactly one vision grid (grid_thws shape[0]==1). Aggregated multi-image inputs must be split upstream (EPD encode server) before feature extraction, because DP owner assignment and CUDA-IPC lease accounting are per-item.","triggerScenarios":"A single MultimodalDataItem whose model_specific_data['grid_thws'] has more than one row reaches get_image_feature(), i.e. batching images into one item instead of splitting them per image.","commonSituations":"Upstream aggregation bug in a custom multimodal pipeline; EPD encode server change that stopped splitting encoder inputs per image; multi-image prompts collapsed into one item.","solutions":["Split aggregated encoder inputs into one MultimodalDataItem per image before calling get_image_feature()","Fix the upstream EPD encode server to emit per-image items","Add an assertion in your pipeline that len(grid_thws)==1 per item"],"exampleFix":"// before\nitems = [aggregate_into_one_item(images)]\nfeats = model.get_image_feature(items)\n\n// after\nitems = [one_item_per_image(img) for img in images]\nfeats = model.get_image_feature(items)","handlingStrategy":"validation","validationCode":"for item in items:\n    grids = item.model_specific_data[\"grid_thws\"]\n    assert grids.shape[0] == 1, \"split aggregated items before get_image_feature()\"","typeGuard":"def items_are_split(items) -> bool:\n    return all(i.model_specific_data[\"grid_thws\"].shape[0] == 1 for i in items)","tryCatchPattern":"try:\n    model.get_image_feature(items)\nexcept ValueError as e:\n    if \"one vision grid\" in str(e):\n        items = split_items_per_image(items)\n        return model.get_image_feature(items)\n    raise","preventionTips":["Emit one MultimodalDataItem per image at ingestion","Add pipeline-level assertions on per-item grid count before DP owner assignment"],"tags":["kimi-k3","multimodal","grid-thws","data-shape"],"backgroundTag":"input-aggregation-validation-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}