sgl-project/sglang · error · ValueError

Step3-VL image item is missing num_patches.

Error message

Step3-VL image item is missing num_patches.

What it means

Step3-VL's get_image_feature requires each multimodal item's model_specific_data to contain num_patches (how many crops/patches the processor produced per image). Missing metadata means patch feature assembly is impossible, so it raises ValueError per item.

Source

Thrown at python/sglang/srt/models/step3_vl.py:822

        image_features = self.vit_downsampler(image_features)
        image_features = self.vit_downsampler2(image_features)
        n_dim = image_features.size(1)
        image_features = image_features.view(B, n_dim, -1).permute(0, 2, 1)
        image_features = self.vit_large_projector(image_features)
        return image_features

    def get_image_feature(self, items: List[MultimodalDataItem]) -> torch.Tensor:
        # Phase 1: Collect thumbnails and patches separately (different resolutions).
        all_thumbnails = []
        all_patches = []
        # Per-item metadata: (thumb_count, num_patches_list, patch_count)
        item_metadata = []

        for item in items:
            pixel_values = item.feature.type(self.vision_model.dtype)
            num_patches = item.model_specific_data.get("num_patches")
            if num_patches is None:
                raise ValueError("Step3-VL image item is missing num_patches.")
            if isinstance(num_patches, torch.Tensor):
                num_patches = [int(x) for x in num_patches.flatten().cpu().tolist()]
            elif isinstance(num_patches, (list, tuple)):
                num_patches = [
                    int(x.item()) if isinstance(x, torch.Tensor) else int(x)
                    for x in num_patches
                ]
            else:
                num_patches = [int(num_patches)]

            patch_pixel_values = item.model_specific_data.get(
                "patch_pixel_values", None
            )
            if patch_pixel_values is not None and patch_pixel_values.shape[0] == 0:
                patch_pixel_values = None
            if patch_pixel_values is not None:
                patch_pixel_values = patch_pixel_values.type(
                    self.vision_model.dtype

View on GitHub (pinned to 0132848349)

Solutions

1. Update sglang so processor and model code are from the same release
2. If building items manually, set item.model_specific_data['num_patches'] to the per-image patch counts from preprocessing
3. Regenerate the multimodal inputs with the server's own preprocessing path

Example fix

# before
item.model_specific_data = {}
# after
item.model_specific_data = {"num_patches": num_patches_list}

for item in items:
    assert item.model_specific_data.get("num_patches") is not None, item

When it happens

Trigger: Passing image items whose model_specific_data lacks 'num_patches' — usually because the processor/tokenizer version predates or differs from the model implementation that consumes it.

Common situations: Mixing sglang versions (processor updated separately from model file), custom multimodal input pipelines that construct EmbeddingItems manually, or serving step3-vl with an outdated processor config.

Related errors

• flattened_bucket 'metadata' must be a list.
• Step3-VL image item has num_patches > 0 but no patch_pixel_v
• Step3-VL image item is missing num_patches.
• Expected CHW image tensor, got shape {shape}
• Expected CHW image tensor with 1 or 3 channels, got shape {s

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a334c27b89de3adb. Report an issue: GitHub.