sgl-project/sglang · error · ValueError

Kimi-K3 deferred feature length does not match image grids

Error message

Kimi-K3 deferred feature length does not match image grids

What it means

Raised in Kimi-K3's multimodal feature materialization (kimi_k3.py:3482) when deferred image features arrive whose total patch count (computed from each selected image's grid_thw on host) does not equal the number of rows in the incoming pixel_values tensor. This is an internal consistency check between the preprocessor's grid metadata and the batched visual embeddings. It signals that the vision encoder's patch-packing got out of sync with the grid bookkeeping, usually due to request batching/broadcast mismatches or a corrupted mm-processor cache.

Source

Thrown at python/sglang/srt/models/kimi_k3.py:3482

                    elif backend == "cpu":
                        from sglang.srt.multimodal.kimi_k3_image_processing import (
                            materialize_kimi_k3_cpu_features,
                        )

                        pixel_values = materialize_kimi_k3_cpu_features(
                            group_items, self._encoder_image_processor
                        )
                    else:
                        raise ValueError(
                            f"Unsupported Kimi-K3 deferred preprocessing backend: {backend}"
                        )

                    patch_counts = [
                        int(grid_thws_host[index].prod().item())
                        for index in global_indices
                    ]
                    if sum(patch_counts) != pixel_values.shape[0]:
                        raise ValueError(
                            "Kimi-K3 deferred feature length does not match image grids"
                        )
                    for index, feature in zip(
                        indices, pixel_values.split(patch_counts), strict=True
                    ):
                        materialized[index] = feature

                return materialize_multimodal_features(
                    materialized,
                    device=device,
                    dtype=target_dtype,
                )

            features = []
            for item in selected_items:
                if not isinstance(item.feature, torch.Tensor):
                    raise TypeError(
                        "Kimi-K3 image feature must be a torch.Tensor, "

View on GitHub (pinned to 0132848349)

Solutions

Ensure each request's images and grid_thw metadata travel together unmodified (no manual slicing/rebatching of mm inputs)
Clear any multimodal cache (restart server / disable mm cache) so grid metadata is recomputed
Reproduce single-image vs batched requests to isolate the offending item and check its grid_thw
If persistent, report with a repro script - the invariant between processor and encoder is broken

Defensive patterns

Strategy: validation

Validate before calling

total = sum(int(g.prod().item()) for g in grids)
assert total == pixel_values.shape[0], f"{total} != {pixel_values.shape[0]}"

Prevention

Keep image payloads and their grid_thw metadata atomically paired through any batching
Avoid hand-slicing multimodal batches; let the scheduler batch mm inputs
Pin model and processor to the same SGLang version

When it happens

Trigger: Calling get_image_feature -> materialize_item_features with a batch where sum(prod(grid_thws_host[i]) for selected deferred items) != pixel_values.shape[0]. Happens with mixed deferred/cached image batches, sliced/re-batched req inputs, or a stale grid_thws cache entry.

Common situations: Custom multi-image workloads, resume/caching of mm inputs, chunked prefill re-batching, or version mismatch between processor and model code after upgrading SGLang.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/81d776dc4607ff6b. Report an issue: GitHub.