sgl-project/sglang · error · ValueError

Only gate value of 1 is supported for int type, but got {gat

Error message

Only gate value of 1 is supported for int type, but got {gate}

What it means

forward_cuda of the diffusion residual norm accepts an int gate only as the neutral value 1 (no gating); any other int (e.g. 0) cannot be expressed by the fused kernel and is rejected before launch.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/layernorm.py:622

        shift: torch.Tensor,
        scale: torch.Tensor,
    ) -> tuple[torch.Tensor, torch.Tensor]:
        if residual.numel() == 0 or x.numel() == 0:
            return self.forward_native(residual, x, gate, shift, scale)

        if x.shape[-1] % 256 != 0 or x.shape[-1] > 8192:
            import warnings

            warnings.warn(
                "FusedScaleResidualNormScaleShift cuda not available, using native fallback",
                stacklevel=2,
            )
            return self.forward_native(residual, x, gate, shift, scale)

        from sglang.kernels.ops.diffusion import fused_scale_residual_norm_scale_shift

        if isinstance(gate, int) and gate != 1:
            raise ValueError(
                f"Only gate value of 1 is supported for int type, but got {gate}"
            )

        return fused_scale_residual_norm_scale_shift(
            residual.contiguous(),
            x.contiguous(),
            gate.contiguous() if isinstance(gate, torch.Tensor) else None,
            _ensure_contiguous(getattr(self.norm, "weight", None)),
            _ensure_contiguous(getattr(self.norm, "bias", None)),
            scale.contiguous(),
            shift.contiguous(),
            self.norm_type,
            self.eps,
        )

    def forward_hip(
        self,
        residual: torch.Tensor,

View on GitHub (pinned to 0132848349)

Solutions

1. Pass gate=1 for ungated forward
2. Pass the gate as a tensor (shape [batch, 1, inner_dim]) when it isn't exactly 1
3. Branch: use native path (`self.forward_native(...)`) when gate is an int != 1

Example fix

# before
out = layer.forward_cuda(residual, x, gate=0, shift=s, scale=sc)
# after
out = layer.forward_cuda(residual, x, gate=1, shift=s, scale=sc)

if isinstance(gate, int) and gate != 1:
    gate = 1  # or raise early with a clear message

def is_valid_gate(g) -> bool:\n    return isinstance(g, torch.Tensor) or (isinstance(g, int) and g == 1)

When it happens

Trigger: Calling forward_cuda with gate as an int != 1; the native fallback path already ran for non-kernel cases, so this specific check guards the fused_scale_residual_norm_scale_shift launch.

Common situations: Code passing a learned gate as an int scalar; using 0 to 'disable' gating — instead pass gate=1 or a tensor; converting a tensor gate of value 1.0 to int 1 is fine, other values are not.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Gate type {type(gate)} not supported
• rmsnorm_hf: unsupported hidden_size={hidden_size} (must be a
• No frames were recorded
• The pointers must be multiple of 16 bytes.
• The last dimension ({input.shape[-1]}) x itemsize ({input.dt

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c652ec1e811746a3. Report an issue: GitHub.