sgl-project/sglang · error · NotImplementedError

MM inputs where only some items are precomputed.

Error message

MM inputs where only some items are precomputed.

What it means

In the multimodal scheduling path, if some items in a multi-item MM request carry precomputed embeddings and others do not, SGLang cannot mix precomputed and to-be-computed features and raises NotImplementedError. All items must be uniformly precomputed or uniformly raw.

Source

Thrown at python/sglang/srt/managers/mm_schedule.py:115

            chunk = None
        else:
            req_embeddings = torch.concat(
                [item.precomputed_embeddings for item in items_per_req]
            )
            chunk, _, _ = get_embedding_chunk(
                embedding=req_embeddings,
                extend_prefix_len=prefix_length[i],
                extend_seq_len=extend_len,
                items_offset=items_offset,
            )

        if chunk is None and len(items_per_req) > 1:
            return None
        precomputed_embeddings.append(chunk)

    if any(feature is not None for feature in precomputed_embeddings):
        if not all(feature is not None for feature in precomputed_embeddings):
            raise NotImplementedError(
                "MM inputs where only some items are precomputed."
            )

        # Normalize device across chunks before concat.
        target_device = next(
            (t.device for t in precomputed_embeddings if t.is_cuda),
            precomputed_embeddings[0].device,
        )
        precomputed_embeddings = [
            t if t.device == target_device else t.to(target_device, non_blocking=True)
            for t in precomputed_embeddings
        ]
        result = torch.concat(precomputed_embeddings)
        # some models embedding is 3-dim, reshape it to 2-dim (similar to get_embedding_chunk)
        result = result.reshape(-1, result.shape[-1])
        return result
    return None

View on GitHub (pinned to 0132848349)

Solutions

1. Compute and include embeddings for ALL items in the request, or none of them
2. Audit the embedding generation step for silent per-item failures
3. For mixed workloads, split into separate requests: fully-precomputed and fully-raw

Example fix

// before
req = {'image_data': [img1, img2], 'precomputed': {'embeds': [e1]}}
// after
req = {'image_data': [img1, img2], 'precomputed': {'embeds': [e1, e2]}}

embeds = precomputed.get('embeds')
if embeds is not None:
    assert len([e for e in embeds if e is not None]) in (0, n_mm_items)

try:
    get_embedding_and_mask(...)
except NotImplementedError:
    # fall back: send raw inputs without precomputed embeddings
    ...

When it happens

Trigger: A single request with multiple images where only some entries in the embedding payload are present (chunk None with len(items_per_req)>1 skips single-item fallback); partial caching of precomputed image embeddings.

Common situations: Building precomputed-embedding payloads where one image lacks its embedding due to an upstream miss; version change that altered embedding payload requirements for multi-image requests.

Related errors

• {selection_error}{component_suffix}
• No compatible attention backend is available{component_suffi
• quantize_and_serve functionality is currently disabled due t
• gRPC encode only supports IMAGE modality, got: {non_image}
• Short-conv hybrid models (ZAYA1 CCA, LFM2 / LFM2-MoE) are no

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d9426b4341a30ef4. Report an issue: GitHub.