sgl-project/sglang · error · ValueError

Gate type {type(gate)} not supported

Error message

Gate type {type(gate)} not supported

What it means

forward_native of the residual-scale-shift norm handles gate as tensor or int(==1); any other type (str, float, None, list) reaches the else branch and raises 'Gate type ... not supported'.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/layernorm.py:708

    ) -> tuple[torch.Tensor, torch.Tensor]:
        # x.shape: [batch_size, seq_len, inner_dim]
        if isinstance(gate, int):
            # used by cross-attention, should be 1
            assert gate == 1
            residual_output = residual + x
        elif isinstance(gate, torch.Tensor):
            if gate.dim() == 4:
                # gate.shape: [batch_size, num_frames, 1, inner_dim]
                num_frames = gate.shape[1]
                frame_seqlen = x.shape[1] // num_frames
                residual_output = residual + (
                    x.unflatten(dim=1, sizes=(num_frames, frame_seqlen)) * gate
                ).flatten(1, 2)
            else:
                # gate.shape: [batch_size, 1, inner_dim]
                residual_output = residual + x * gate
        else:
            raise ValueError(f"Gate type {type(gate)} not supported")
        normalized = self.norm(residual_output)
        modulated = fuse_scale_shift_kernel(normalized, scale, shift)
        return modulated, residual_output

    def forward_npu(
        self,
        residual: torch.Tensor,
        x: torch.Tensor,
        gate: torch.Tensor | int,
        shift: torch.Tensor,
        scale: torch.Tensor,
    ) -> tuple[torch.Tensor, torch.Tensor]:
        # x.shape: [batch_size, seq_len, inner_dim]
        if isinstance(gate, int):
            # used by cross-attention, should be 1
            assert gate == 1
            residual_output = residual + x
        elif isinstance(gate, torch.Tensor):

View on GitHub (pinned to 0132848349)

Solutions

1. Coerce: `gate = torch.as_tensor(gate)` or `int(gate)` (must be 1) before calling
2. Default gate to 1 when gating is disabled
3. Check the caller chain for None/float gate leaks

Example fix

# before
modulated, res = layer.forward_native(residual, x, gate=1.0, shift=s, scale=sc)
# after
modulated, res = layer.forward_native(residual, x, gate=1, shift=s, scale=sc)

if not isinstance(gate, (torch.Tensor, int)):
    gate = torch.as_tensor(gate, dtype=x.dtype) if gate is not None else 1

def is_valid_gate(g) -> bool:\n    return isinstance(g, torch.Tensor) or (isinstance(g, int) and g == 1)

try:\n    out = layer.forward_native(residual, x, gate, shift, scale)\nexcept ValueError as e:\n    if 'Gate type' in str(e):\n        out = layer.forward_native(residual, x, torch.as_tensor(gate), shift, scale)

When it happens

Trigger: Calling forward_native (directly or via forward_cuda/cpu/hip fallback) with a gate that is neither a torch.Tensor nor int — e.g. a float 1.0 or None.

Common situations: Gate loaded from config as float; a code path where gate is optional and None is passed instead of 1; numpy scalar instead of int/tensor.

Understand the failure class

Background: Invalid argument type errors: "must be of type string", "expected X, got Y", and ERR_INVALID_ARG_TYPE explained — this error's family across 15 libraries.

Related errors

• Only gate value of 1 is supported for int type, but got {gat
• `b` must be 2D (got b.ndim={b.ndim}).
• spt must be a bool when provided
• attn_sink requires topk_length to be provided as well
• missing value for {a} (expected e.g. `{a} 2,4`)

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/81cf3fa6a4d5287b. Report an issue: GitHub.