sgl-project/sglang · error · ValueError

Grid dim ({_mm_grid_attrs[modality]}) not found in {mm_input

Error message

Grid dim ({_mm_grid_attrs[modality]}) not found in {mm_inputs}

What it means

Thrown by the multimodal preprocessor when it cannot find any of the expected grid-shape keys (e.g. image_grid_thw, grid_thws, image_grid_hws, video_grid_thw) in the mm_inputs dict for the given modality. The grid metadata is required to compute patch counts and token counts for slicing pixel_values. Model-specific key preference applies (kimi_k25/k3 prefer grid_thws, kimi_vl prefers image_grid_hws), but if none of the candidate keys is present and non-None the lookup fails.

Source

Thrown at python/sglang/srt/disaggregation/encoder/preprocessor.py:659

            return (input_length - 2) // 2 + 1

    def _get_mm_grid_dim(self, mm_inputs: dict, modality: Modality):
        # Kimi K2.5/K3 vision processors only emit `grid_thws`; prefer it over generic keys
        # so we never pick a mis-typed or stale `image_grid_hws` field from kwargs.
        attrs = _mm_grid_attrs[modality]
        model_type = (self.model_type or "").lower()
        if modality == Modality.IMAGE:
            # Kimi K2.5/K3 emit grid_thws, while Kimi-VL emits image_grid_hws.
            # Other model types keep the generic attr order above.
            if model_type in ("kimi_k25", "kimi_k3"):
                attrs = ("grid_thws", "image_grid_thw", "image_grid_hws")
            elif model_type == "kimi_vl":
                attrs = ("image_grid_hws", "image_grid_thw", "grid_thws")

        for attr in attrs:
            if attr in mm_inputs and mm_inputs[attr] is not None:
                return _convert(mm_inputs[attr])
        raise ValueError(
            f"Grid dim ({_mm_grid_attrs[modality]}) not found in {mm_inputs}"
        )

    def get_num_patches(
        self, grid: Union[torch.Tensor, List[int]], modality: Modality
    ) -> int:
        """Calculate number of raw patches (before merge/sampling). Used for pixel_values slicing."""
        if modality == Modality.AUDIO:
            return int(grid.item())
        if self.model_type == "kimi_vl" and modality == Modality.IMAGE:
            h, w = self._kimi_hw_from_patch_grid(grid)
            return h * w
        return int(grid[0] * grid[1] * grid[2])

    @staticmethod
    def _kimi_hw_from_patch_grid(
        grid: Union[torch.Tensor, np.ndarray, List[int], Tuple[int, ...]],
    ) -> Tuple[int, int]:

View on GitHub (pinned to 0132848349)

Solutions

1. Inspect the failing mm_inputs dict (the message prints it) and confirm which grid key your processor actually emits.
2. Ensure the output of the HF vision processor (image_grid_thw / grid_thws / video_grid_thw etc.) is passed through unmodified into mm_inputs.
3. If a custom preprocessor drops keys, add the missing key back or map the new name to the one expected for your model_type.
4. Check that the value is not None — a None value is treated as missing.

Example fix

// before
mm_inputs = {"pixel_values": pv}  # grid key lost
// after
mm_inputs = {"pixel_values": pv, "image_grid_thw": proc_out["image_grid_thw"]}

from sglang.srt.disaggregation.encoder.preprocessor import _mm_grid_attrs

def has_grid_dim(mm_inputs: dict, modality) -> bool:
    return any(
        k in mm_inputs and mm_inputs[k] is not None
        for k in _mm_grid_attrs[modality]
    )

def has_valid_grid(mm_inputs: dict) -> bool:
    return isinstance(mm_inputs, dict) and any(
        isinstance(v, (list, tuple)) and len(v) and all(v)
        for k, v in mm_inputs.items()
        if "grid" in k or k.endswith("_thw") or k.endswith("_hws")
    )

try:
    grid = prep._get_mm_grid_dim(mm_inputs, modality)
except ValueError as e:
    raise HTTPException(400, f"multimodal input missing grid metadata: {e}") from e

When it happens

Trigger: Calling process_mm_items (or the encoder pipeline that drives it) with mm_inputs that lack the processor-emitted grid tensor, e.g. passing only pixel_values without image_grid_thw/grid_thws; or a video/audio item where the corresponding grid key was dropped, renamed by a newer transformers version, or serialized to None during disaggregated handoff.

Common situations: Upgrading transformers so the vision processor emits a differently-named grid key; hand-crafting mm_inputs dicts in tests or custom clients; deserializing grid metadata that was lost/None'd over the wire; wrong modality key set for the model type.

Understand the failure class

Background: "Missing required field" and "field is required" errors: why libraries reject payloads that omit mandatory fields — this error's family across 20 libraries.

Related errors

• Invalid grid metadata for kimi image tokens: {vals} (expecte
• Invalid Kimi image grid metadata: {values}; expected [h, w]
• Feature attrs ({_mm_feature_attrs[modality]}) not found in {
• Unsupported grid type for kimi image tokens: {type(grid_thw)
• The number of image placeholders exceeds img_grid_thw entrie

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/389292a71a1653f0. Report an issue: GitHub.