sgl-project/sglang · error · ValueError

Kimi-K3 expects one vision grid per MultimodalDataItem; spli

Error message

Kimi-K3 expects one vision grid per MultimodalDataItem; split aggregated encoder inputs before get_image_feature()

What it means

get_image_feature() requires each MultimodalDataItem to carry exactly one vision grid (grid_thws shape[0]==1). Aggregated multi-image inputs must be split upstream (EPD encode server) before feature extraction, because DP owner assignment and CUDA-IPC lease accounting are per-item.

Source

Thrown at python/sglang/srt/models/kimi_k3.py:3368

            mm_data,
            image_processor,
            use_gpu_preprocessing=use_gpu_preprocessing,
        )

    def get_image_feature(self, items: List[MultimodalDataItem]) -> torch.Tensor:
        device = self.vision_tower.device
        target_dtype = self.vision_tower.patch_embed.proj.weight.dtype
        image_grid_thws = []
        for item in items:
            grid_thw = item.model_specific_data.get("image_grid_thw")
            if grid_thw is None:
                grid_thw = item.model_specific_data["grid_thws"]
            if grid_thw.shape[0] != 1:
                # One item must carry exactly one logical image so the DP
                # owner assignment and the bounded CUDA-IPC lease accounting
                # stay per-item; aggregated encoder inputs are split upstream
                # (EPD encode server) before reaching this point.
                raise ValueError(
                    "Kimi-K3 expects one vision grid per MultimodalDataItem; "
                    "split aggregated encoder inputs before get_image_feature()"
                )
            image_grid_thws.append(grid_thw)
        grid_thws_host = torch.concat(image_grid_thws, dim=0).cpu()
        grid_thw_list = grid_thws_host.tolist()

        def materialize_item_features(image_indices: List[int]) -> torch.Tensor:
            """Materialize only the images assigned to this vision-DP rank."""
            from sglang.srt.multimodal.encoder_preprocessing import (
                LOCAL_PREPROCESSED_KEY,
            )

            # Match the configured TP consumer count captured when the
            # tokenizer creates MmItemMemoryPool. A live attention subgroup
            # size could leave acknowledgements missing and strand the lease.
            ipc_consumer_count = max(get_parallel().tp_size, 1)
            device_index = device.index

View on GitHub (pinned to 0132848349)

Solutions

1. Split aggregated encoder inputs into one MultimodalDataItem per image before calling get_image_feature()
2. Fix the upstream EPD encode server to emit per-image items
3. Add an assertion in your pipeline that len(grid_thws)==1 per item

Example fix

// before
items = [aggregate_into_one_item(images)]
feats = model.get_image_feature(items)

// after
items = [one_item_per_image(img) for img in images]
feats = model.get_image_feature(items)

for item in items:
    grids = item.model_specific_data["grid_thws"]
    assert grids.shape[0] == 1, "split aggregated items before get_image_feature()"

def items_are_split(items) -> bool:
    return all(i.model_specific_data["grid_thws"].shape[0] == 1 for i in items)

try:
    model.get_image_feature(items)
except ValueError as e:
    if "one vision grid" in str(e):
        items = split_items_per_image(items)
        return model.get_image_feature(items)
    raise

When it happens

Trigger: A single MultimodalDataItem whose model_specific_data['grid_thws'] has more than one row reaches get_image_feature(), i.e. batching images into one item instead of splitting them per image.

Common situations: Upstream aggregation bug in a custom multimodal pipeline; EPD encode server change that stopped splitting encoder inputs per image; multi-image prompts collapsed into one item.

Related errors

• Kimi-K3 encoder mode supports image input only
• Kimi-K3 cannot mix local preprocessed and deferred images
• Kimi-K3 image feature must be a torch.Tensor, got {type(item
• Kimi-K3 deferred feature length does not match image grids
• Kimi-K3 image processor is missing deferred-preprocessing co

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/5064476f544104b9. Report an issue: GitHub.