sgl-project/sglang · error · ValueError

Unknown visual_type: {visual_type}

Error message

Unknown visual_type: {visual_type}

What it means

Raised in _flatten_visual_inputs when visual_type doesn't match any known branch of the flatten dispatch. The function reshapes visual patches for image vs video visual types; anything else (unknown string, typo'd enum) falls through to this ValueError.

Source

Thrown at python/sglang/srt/multimodal/processors/mimo_v2.py:1296

            position_ids=position_ids,
            rope_deltas=rope_deltas,
            extra=extra,
        )

    def _flatten_visual_inputs(self, visual: torch.Tensor, visual_type: str):
        if visual_type == "image":
            resized_height, resized_width = visual.shape[-2:]
            patches = visual.unsqueeze(0).repeat(self.temporal_patch_size, 1, 1, 1)
        elif visual_type == "video" or visual_type == "video_audio":
            assert (
                len(visual)
                % (self.temporal_compression_ratio * self.temporal_patch_size)
                == 0
            )
            patches = visual
            resized_height, resized_width = patches.shape[-2:]
        else:
            raise ValueError(f"Unknown visual_type: {visual_type}")

        channel = patches.shape[1]
        grid_t = patches.shape[0] // self.temporal_patch_size
        grid_h, grid_w = (
            resized_height // self.patch_size,
            resized_width // self.patch_size,
        )
        patches = patches.contiguous().view(
            grid_t,
            self.temporal_patch_size,
            channel,
            grid_h // self.merge_size,
            self.merge_size,
            self.patch_size,
            grid_w // self.merge_size,
            self.merge_size,
            self.patch_size,
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Check the visual_type value in the message against the keys handled in _flatten_visual_inputs in your sglang version
2. Ensure sglang and any custom model code are the same version so visual type constants match
3. Route only image/video visuals through this path; audio has its own preprocessing

assert visual_type in ('image', 'video'), f'unknown visual_type {visual_type!r} for this sglang version'

def is_known_visual_type(t: str) -> bool:
    return t in {'image', 'video'}  # keep in sync with _flatten_visual_inputs branches

try:
    flat = proc._flatten_visual_inputs(visual, visual_type)
except ValueError as e:
    if 'Unknown visual_type' in str(e):
        raise Http400(f'visual_type {visual_type!r} not supported by model version')
    raise

When it happens

Trigger: Calling _flatten_visual_inputs (directly or via preprocess_for_encoder/process_video/_process_image_content) with a visual_type string that isn't one of the handled image/video keys — e.g. 'audio', an outdated name, or a custom value.

Common situations: Version skew where a newer/older sglang defines different visual_type keys than the model code passing them; internal refactors renaming visual type constants; custom multimodal extensions introducing new visual kinds without extending the dispatcher.

Related errors

• Unsupported runner backend: %s
• Unsupported modality for EPD preprocessing: {modality}
• unsupported input for causal Conv3D cat/pad CUDA
• unsupported input for usp_merge_heads CUDA
• unsupported input for modulate_scale_shift CUDA

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6119d871c349d3b9. Report an issue: GitHub.