sgl-project/sglang · error · ValueError

Kimi-K3 deferred GPU preprocessing produced wrong grids

Error message

Kimi-K3 deferred GPU preprocessing produced wrong grids

What it means

After deferred GPU preprocessing, the produced grid_thws must exactly equal the expected grids (grid_thws_host for those indices); a mismatch raises ValueError. This is a self-check that the GPU resize/patching pipeline reproduced the expected vision grids.

Source

Thrown at python/sglang/srt/models/kimi_k3.py:3461

                        image_scale, image_bias = normalization_tensors(
                            first_config.image_mean,
                            first_config.image_std,
                            device,
                        )
                        pixel_values, produced_grids = _gpu_preprocess_images(
                            [item.feature for item in group_items],
                            [config.resize_config for config in group_configs],
                            image_scale,
                            image_bias,
                            self.vision_tower.patch_size,
                            to_chw=lambda image: to_chw_uint8(image, device=device),
                            post_resize=lambda x: fill_transparent_bg(
                                x, first_config.transparent_bg_config
                            ),
                        )
                        expected_grids = grid_thws_host[global_indices]
                        if not torch.equal(produced_grids.cpu(), expected_grids):
                            raise ValueError(
                                "Kimi-K3 deferred GPU preprocessing produced wrong grids"
                            )
                    elif backend == "cpu":
                        from sglang.srt.multimodal.kimi_k3_image_processing import (
                            materialize_kimi_k3_cpu_features,
                        )

                        pixel_values = materialize_kimi_k3_cpu_features(
                            group_items, self._encoder_image_processor
                        )
                    else:
                        raise ValueError(
                            f"Unsupported Kimi-K3 deferred preprocessing backend: {backend}"
                        )

                    patch_counts = [
                        int(grid_thws_host[index].prod().item())
                        for index in global_indices

View on GitHub (pinned to 0132848349)

Solutions

Verify resize config and transparent_bg_config match the HF reference processor
Update/re-pin the GPU preprocessing kernel so its grid computation matches the reference
As a workaround, route those items through the CPU backend (materialize_kimi_k3_cpu_features)
Report with the offending image and configs if it persists after config alignment

Example fix

// before
backend = "gpu"  # raises on grid mismatch

// after
backend = "cpu"  # deterministic reference path while GPU grids are fixed

Defensive patterns

Strategy: fallback

Try / catch

try:
    feats = model.get_image_feature(items)
except ValueError as e:
    if "wrong grids" in str(e):
        mark_items_cpu_backend(items)
        feats = model.get_image_feature(items)  # CPU reference path
    else:
        raise

Prevention

Pin the GPU preprocessing kernel version matched to the reference processor
Keep resize and transparent_bg_config identical to the HF processor config
Run golden-image tests comparing GPU vs CPU grids on each release

When it happens

Trigger: Deferred GPU preprocessing (backend='gpu') runs for a group and torch.equal(produced_grids, expected_grids) is False — e.g. resize/post-resize (transparent background fill) behaving differently than the reference processor.

Common situations: Changes to image resize parameters, transparent_bg_config mismatches, GPU preprocessing kernel/version drift, or non-deterministic resizing on unusual image sizes.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/60482d5a321273df. Report an issue: GitHub.