sgl-project/sglang · error · ValueError

You must specify exactly one of input_ids or inputs_embeds

Error message

You must specify exactly one of input_ids or inputs_embeds

What it means

Gemma2 encoder forward requires exactly one of input_ids or inputs_embeds. The XOR check rejects calls with both (ambiguous) or neither (nothing to embed).

Source

Thrown at python/sglang/multimodal_gen/runtime/models/encoders/gemma2.py:340

            ]
        )

        self.norm = Gemma2RMSNorm(arch.hidden_size, eps=arch.rms_norm_eps)

    def get_input_embeddings(self, input_ids: torch.Tensor) -> torch.Tensor:
        return self.embed_tokens(input_ids) * self.embed_scale

    def forward(
        self,
        input_ids: torch.Tensor | None = None,
        position_ids: torch.Tensor | None = None,
        attention_mask: torch.Tensor | None = None,
        inputs_embeds: torch.Tensor | None = None,
        output_hidden_states: bool | None = None,
        **kwargs,
    ) -> BaseEncoderOutput:
        if (input_ids is None) ^ (inputs_embeds is not None):
            raise ValueError(
                "You must specify exactly one of input_ids or inputs_embeds"
            )

        output_hidden_states = (
            output_hidden_states
            if output_hidden_states is not None
            else getattr(self.config.arch_config, "output_hidden_states", False)
        )

        if inputs_embeds is not None:
            hidden_states = inputs_embeds
        else:
            hidden_states = self.get_input_embeddings(input_ids)

        if position_ids is None:
            position_ids = torch.arange(
                0, hidden_states.shape[1], device=hidden_states.device
            ).unsqueeze(0)

View on GitHub (pinned to 0132848349)

Solutions

1. Pass exactly one of input_ids (token IDs) or inputs_embeds (pre-embedded floats)
2. In multimodal flows, pass inputs_embeds where image features are already merged into the token embeddings

Example fix

# before
out = model(input_ids=ids, inputs_embeds=emb)
# after
out = model(inputs_embeds=emb)

assert (input_ids is None) != (inputs_embeds is None), "pass exactly one of input_ids / inputs_embeds"

def exactly_one(a, b) -> bool:
    return (a is None) != (b is None)

When it happens

Trigger: Calling forward with both input_ids and inputs_embeds, or with neither (e.g. only attention_mask/pixel_values).

Common situations: Multimodal code that injects image embeddings via inputs_embeds while still passing input_ids; refactoring away from input_ids without removing it from the call site.

Related errors

• unsupported input for causal Conv3D cat/pad CUDA
• unsupported input for usp_merge_heads CUDA
• unsupported input for modulate_scale_shift CUDA
• unsupported input for LTX2 QKNorm split-RoPE CUDA
• kv-canary: expected input tensors must be None when enable_w

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d6f19db6eb4104c5. Report an issue: GitHub.