sgl-project/sglang · error · ValueError

You must specify exactly one of input_ids or inputs_embeds

Error message

You must specify exactly one of input_ids or inputs_embeds

What it means

The Mistral3 text model forward requires exactly one of input_ids or inputs_embeds. The XOR check rejects both being None (nothing to embed) and both being set (ambiguous input).

Source

Thrown at python/sglang/multimodal_gen/runtime/models/encoders/mistral_3.py:432

        self.norm = MistralRMSNorm(config.hidden_size, eps=config.rms_norm_eps)
        self.rotary_emb = MistralRotaryEmbedding(config=config)
        self.gradient_checkpointing = False
        self.post_init()

    def forward(
        self,
        input_ids: Optional[torch.LongTensor] = None,
        attention_mask: Optional[torch.Tensor] = None,
        position_ids: Optional[torch.LongTensor] = None,
        past_key_values: Optional[Cache] = None,
        inputs_embeds: Optional[torch.FloatTensor] = None,
        use_cache: Optional[bool] = None,
        cache_position: Optional[torch.LongTensor] = None,
        output_hidden_states: Optional[bool] = None,
        **kwargs,
    ) -> BaseModelOutputWithPast:
        if (input_ids is None) ^ (inputs_embeds is not None):
            raise ValueError(
                "You must specify exactly one of input_ids or inputs_embeds"
            )

        if inputs_embeds is None:
            inputs_embeds = self.embed_tokens(input_ids)

        if use_cache and past_key_values is None:
            past_key_values = DynamicCache(config=self.config)

        if cache_position is None:
            past_seen_tokens = (
                past_key_values.get_seq_length() if past_key_values is not None else 0
            )
            cache_position = torch.arange(
                past_seen_tokens,
                past_seen_tokens + inputs_embeds.shape[1],
                device=inputs_embeds.device,
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Pass exactly one: either input_ids (ids get embedded internally) or inputs_embeds
2. In multimodal paths, set input_ids=None when supplying merged inputs_embeds

Example fix

# before
out = model(input_ids=ids, inputs_embeds=embeds)
# after
out = model(inputs_embeds=embeds)  # input_ids omitted

assert (input_ids is None) != (inputs_embeds is None), "pass exactly one of input_ids/inputs_embeds"

def has_exactly_one_input(ids, embeds) -> bool:
    return (ids is None) ^ (embeds is None)

When it happens

Trigger: Calling Mistral3TextModel.forward with input_ids=None and inputs_embeds=None, or with both provided simultaneously.

Common situations: Adapters that always pass embedded inputs while also forwarding the original ids; multimodal wrappers forgetting to clear input_ids after merging pixel embeddings; None-propagation from upstream preprocessing.

Related errors

• {key}.position_ids is required
• unsupported input for causal Conv3D cat/pad CUDA
• unsupported input for usp_merge_heads CUDA
• unsupported input for modulate_scale_shift CUDA
• unsupported input for LTX2 QKNorm split-RoPE CUDA

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/74107b2978be3720. Report an issue: GitHub.