sgl-project/sglang · error · ValueError

Unsupported activation type {self.glu_act}

Error message

Unsupported activation type {self.glu_act}

What it means

A GLU (gated linear unit) block in phi4mm_utils.py only supports relu, gelu, and swish activation types for its gate. Passing any other glu_type string raises ValueError — note the message mistakenly interpolates self.glu_act, so it prints an activation object, but the real problem is the glu_type config value.

Source

Thrown at python/sglang/srt/models/phi4mm_utils.py:222

        else:
            self.ext_pw_conv_1d = nn.Conv1d(
                input_dim,
                output_dim * 2,
                kernel_size,
                1,
                padding=(kernel_size - 1) // 2,
            )

        if glu_type == "sigmoid":
            self.glu_act = nn.Sigmoid()
        elif glu_type == "relu":
            self.glu_act = nn.ReLU()
        elif glu_type == "gelu":
            self.glu_act = nn.GELU()
        elif glu_type == "swish":
            self.glu_act = Swish()
        else:
            raise ValueError(f"Unsupported activation type {self.glu_act}")

        if bias_in_glu:
            self.b1 = nn.Parameter(torch.zeros(1, output_dim, 1))
            self.b2 = nn.Parameter(torch.zeros(1, output_dim, 1))

    def forward(self, x):
        """
        Args:
            x: torch.Tensor
                input tensor
        """
        # to be consistent with GLULinear, we assume the input always has the
        # #channel (#dim) in the last dimension of the tensor, so need to
        # switch the dimension first for 1D-Conv case
        x = x.permute([0, 2, 1])
        x = self.ext_pw_conv_1d(x)
        if self.glu_type == "bilinear":
            if self.bias_in_glu:

View on GitHub (pinned to 0132848349)

Solutions

1. Set glu_type to one of relu, gelu, or swish in the audio/conformer config
2. If you need silu/mish, map it to swish (they are equivalent) or add the activation as a new branch before the raise

Example fix

# before
GLU(output_dim, input_dim, glu_type="silu")  # ValueError
# after
GLU(output_dim, input_dim, glu_type="swish")  # swish == silu

assert glu_type in ("relu", "gelu", "swish"), f"unsupported glu_type: {glu_type}"

def is_supported_glu_type(t: str) -> bool:
    return t in ("relu", "gelu", "swish")

When it happens

Trigger: Constructing the GLU module (used in the Phi-4-MM audio conformer feed-forward) with glu_type not in {'relu','gelu','swish'} — e.g. 'silu', 'tanh', 'mish', or a typo.

Common situations: Porting a conformer config from another codebase (NeMo uses e.g. 'swish'/'gelu' but other libs use 'silu'); typos or case differences in config files.

Related errors

• Unknown activation function: {act_fn}
• The hpc_ops MoE runner backend runs a plain SiLU-and-mul; it
• Unsupported activation: {self.activation}
• Unsupported activation: {hidden_act}. Only xIELU is supporte
• Unsupported activation: {hidden_act}. Arcee model in SGLang

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c4b3ee962f79c262. Report an issue: GitHub.