sgl-project/sglang · error · ValueError

Kimi-K3 cannot mix local preprocessed and deferred images

Error message

Kimi-K3 cannot mix local preprocessed and deferred images

What it means

materialize_item_features() refuses batches that mix locally preprocessed images (LOCAL_PREPROCESSED_KEY=True) with deferred ones, raising ValueError. Mixing would require handling two incompatible feature paths in one batch, so the model requires all-or-nothing.

Source

Thrown at python/sglang/srt/models/kimi_k3.py:3405

            if device.type == "cuda" and device_index is None:
                device_index = torch.cuda.current_device()

            selected_items = []
            for image_index in image_indices:
                item = items[image_index]
                if device.type == "cuda":
                    item.reconstruct(
                        device_index, ipc_consumer_count=ipc_consumer_count
                    )
                selected_items.append(item)

            locally_preprocessed = [
                item.model_specific_data.get(LOCAL_PREPROCESSED_KEY, False)
                for item in selected_items
            ]
            if any(locally_preprocessed):
                if not all(locally_preprocessed):
                    raise ValueError(
                        "Kimi-K3 cannot mix local preprocessed and deferred images"
                    )
                return materialize_multimodal_features(
                    [item.feature for item in selected_items],
                    device=device,
                    dtype=target_dtype,
                )

            deferred = [
                item.model_specific_data.get(DEFERRED_PREPROCESSING_KEY)
                for item in selected_items
            ]
            if any(config is not None for config in deferred):
                materialized = [None] * len(selected_items)
                deferred_by_backend = {}
                for index, (item, config) in enumerate(zip(selected_items, deferred)):
                    if config is None:
                        if not isinstance(item.feature, torch.Tensor):

View on GitHub (pinned to 0132848349)

Solutions

1. Group items by preprocessed-vs-deferred and call materialization per homogeneous group
2. Make the input pipeline set LOCAL_PREPROCESSED_KEY consistently for all items in a request
3. If using the deferred path, ensure all images in the batch carry deferred configs

Example fix

// before
feats = model.get_image_feature(mixed_items)

// after
local = [i for i in items if i.model_specific_data.get(LOCAL_PREPROCESSED_KEY)]
deferred = [i for i in items if not i.model_specific_data.get(LOCAL_PREPROCESSED_KEY)]
feats = model.get_image_feature(local) + model.get_image_feature(deferred)

flags = [i.model_specific_data.get(LOCAL_PREPROCESSED_KEY, False) for i in items]
assert all(f == flags[0] for f in flags), "do not mix local and deferred items"

def homogeneous_batch(items) -> bool:
    flags = {bool(i.model_specific_data.get(LOCAL_PREPROCESSED_KEY, False)) for i in items}
    return len(flags) == 1

try:
    model.get_image_feature(items)
except ValueError as e:
    if "mix" in str(e):
        groups = group_by_preprocessed_flag(items)
        return [model.get_image_feature(g) for g in groups]
    raise

When it happens

Trigger: A single get_image_feature() call where some selected items have model_specific_data[LOCAL_PREPROCESSED_KEY]=True and others don't.

Common situations: Multi-image requests where one image was preprocessed locally (e.g. CPU path) and others arrived via deferred GPU preprocessing; inconsistent flags set by different branches of the input pipeline.

Related errors

• Kimi-K3 encoder mode supports image input only
• Kimi-K3 expects one vision grid per MultimodalDataItem; spli
• Kimi-K3 image feature must be a torch.Tensor, got {type(item
• Kimi-K3 deferred feature length does not match image grids
• Kimi-K3 image processor is missing deferred-preprocessing co

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b83c7771c03c9f99. Report an issue: GitHub.