sgl-project/sglang · error · TypeError

Unsupported grid type for kimi image tokens: {type(grid_thw)

Error message

Unsupported grid type for kimi image tokens: {type(grid_thw)}

What it means

Kimi multimodal token-count helper only accepts grid_thw of torch.Tensor, np.ndarray, list, or tuple. Any other type (e.g. a string, dict, or custom object) reaches the else branch and raises TypeError. The grid encodes [t,h,w] patch counts used to compute how many image tokens each placeholder expands to.

Source

Thrown at python/sglang/srt/multimodal/processors/kimi_common.py:70

            ),
            dtype=np.int64,
        )
        return int(np.count_nonzero(token_ids == image_token_id))

    def _num_image_tokens_from_grid(
        self, grid_thw: Union[torch.Tensor, np.ndarray, list, tuple]
    ) -> int:
        """Compute Kimi-style image token count from 2D/3D grid metadata."""
        merge_h, merge_w = self.hf_config.vision_config.merge_kernel_size

        if isinstance(grid_thw, torch.Tensor):
            vals = grid_thw.flatten().tolist()
        elif isinstance(grid_thw, np.ndarray):
            vals = grid_thw.reshape(-1).tolist()
        elif isinstance(grid_thw, (list, tuple)):
            vals = list(np.array(grid_thw).reshape(-1).tolist())
        else:
            raise TypeError(
                f"Unsupported grid type for kimi image tokens: {type(grid_thw)}"
            )

        if len(vals) >= 3:
            _t, h, w = vals[-3], vals[-2], vals[-1]
        elif len(vals) == 2:
            _t, h, w = 1, vals[0], vals[1]
        else:
            raise ValueError(
                f"Invalid grid metadata for kimi image tokens: {vals} "
                "(expected [t,h,w] or [h,w])"
            )

        h, w = int(h), int(w)
        return (h * w) // (merge_h * merge_w)

    def _build_kimi_mm_data_from_grids(
        self, prompt, embeddings, **kwargs

View on GitHub (pinned to 0132848349)

Solutions

1. Convert grid_thw to a list/tuple of ints or a torch.Tensor/np.ndarray before calling the API
2. If it arrives as a JSON string, parse it first: json.loads(...) then pass the list
3. Normalize grids at the boundary of your data pipeline with np.asarray(grid, dtype=int)

Example fix

// before
build(grid_thw="[[1,4,4]]")

// after
import json
build(grid_thw=json.loads("[[1,4,4]]"))  # or torch.tensor([[1,4,4]])

import numpy as np
assert isinstance(grid_thw, (torch.Tensor, np.ndarray, list, tuple)), type(grid_thw)
grid_thw = np.asarray(grid_thw)

def is_supported_grid(g) -> bool:
    return isinstance(g, (torch.Tensor, np.ndarray, list, tuple))

try:
    build_from_grids(prompt, grids)
except TypeError as e:
    if "Unsupported grid type" in str(e):
        grids = [np.asarray(json.loads(g)) if isinstance(g, str) else np.asarray(g) for g in grids]
        build_from_grids(prompt, grids)
    else:
        raise

When it happens

Trigger: Calling _build_kimi_mm_data_from_grids or get_mm_data with img_grid_thw passed as a non-array type such as a JSON string, dict, or nested custom object instead of a tensor/array/list.

Common situations: Deserializing grid metadata from JSON without converting back to arrays, passing raw HF processor output that was serialized/round-tripped, or a custom data loader emitting strings.

Related errors

• Grid dim ({_mm_grid_attrs[modality]}) not found in {mm_input
• Invalid Kimi image grid metadata: {values}; expected [h, w]
• Invalid grid metadata for kimi image tokens: {vals} (expecte
• {field_name} must be a tensor, list of tensors, list of sequ
• Incorrect type of pixel values. Got type: {type(pixel_values

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2b85b75b4e69b8cc. Report an issue: GitHub.