sgl-project/sglang · error · ValueError

Step3-VL image item is missing num_patches.

Error message

Step3-VL image item is missing num_patches.

What it means

Step3-VL's get_image_feature requires each multimodal item's model_specific_data to contain num_patches (how many crops/patches the processor produced per image). Missing metadata means patch feature assembly is impossible, so it raises ValueError per item.

Solutions

1. Update sglang so processor and model code are from the same release
2. If building items manually, set item.model_specific_data['num_patches'] to the per-image patch counts from preprocessing
3. Regenerate the multimodal inputs with the server's own preprocessing path

Example fix

# before
item.model_specific_data = {}
# after
item.model_specific_data = {"num_patches": num_patches_list}

for item in items:
    assert item.model_specific_data.get("num_patches") is not None, item

When it happens

Trigger: Passing image items whose model_specific_data lacks 'num_patches' — usually because the processor/tokenizer version predates or differs from the model implementation that consumes it.

Common situations: Mixing sglang versions (processor updated separately from model file), custom multimodal input pipelines that construct EmbeddingItems manually, or serving step3-vl with an outdated processor config.

Related errors

• Expected CHW image tensor, got shape
• Expected CHW image tensor with 1 or 3 channels, got shape
• flattened_bucket 'metadata' must be a list.
• Step3-VL image item has num_patches > 0 but no…
• Step3-VL image item is missing num_patches.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a334c27b89de3adb. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/models/step3_vl.py:822

        image_features = self.vit_downsampler(image_features)
        image_features = self.vit_downsampler2(image_features)
        n_dim = image_features.size(1)
        image_features = image_features.view(B, n_dim, -1).permute(0, 2, 1)
        image_features = self.vit_large_projector(image_features)
        return image_features

    def get_image_feature(self, items: List[MultimodalDataItem]) -> torch.Tensor:
        # Phase 1: Collect thumbnails and patches separately (different resolutions).
        all_thumbnails = []
        all_patches = []
        # Per-item metadata: (thumb_count, num_patches_list, patch_count)
        item_metadata = []

        for item in items:
            pixel_values = item.feature.type(self.vision_model.dtype)
            num_patches = item.model_specific_data.get("num_patches")
            if num_patches is None:
                raise ValueError("Step3-VL image item is missing num_patches.")
            if isinstance(num_patches, torch.Tensor):
                num_patches = [int(x) for x in num_patches.flatten().cpu().tolist()]
            elif isinstance(num_patches, (list, tuple)):
                num_patches = [
                    int(x.item()) if isinstance(x, torch.Tensor) else int(x)
                    for x in num_patches
                ]
            else:
                num_patches = [int(num_patches)]

            patch_pixel_values = item.model_specific_data.get(
                "patch_pixel_values", None
            )
            if patch_pixel_values is not None and patch_pixel_values.shape[0] == 0:
                patch_pixel_values = None
            if patch_pixel_values is not None:
                patch_pixel_values = patch_pixel_values.type(
                    self.vision_model.dtype

View on GitHub (pinned to 0132848349)