sgl-project/sglang · critical · ValueError

Unknown activation function: {act_fn}

Error message

Unknown activation function: {act_fn}

What it means

The LTX-2 caption/text projection supports only gelu_tanh and silu activation in its MLP. An unknown act_fn string fails at init when trying to select the activation module.

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/ltx_2.py:622

        self,
        in_features: int,
        hidden_size: int,
        out_features: int | None = None,
        act_fn: str = "gelu_tanh",
    ) -> None:
        super().__init__()
        if out_features is None:
            out_features = hidden_size

        self.linear_1 = ColumnParallelLinear(
            in_features, hidden_size, bias=True, gather_output=True
        )
        if act_fn == "gelu_tanh":
            self.act_1 = nn.GELU(approximate="tanh")
        elif act_fn == "silu":
            self.act_1 = nn.SiLU()
        else:
            raise ValueError(f"Unknown activation function: {act_fn}")

        self.linear_2 = ColumnParallelLinear(
            hidden_size, out_features, bias=True, gather_output=True
        )

    def forward(self, caption: torch.Tensor) -> torch.Tensor:
        hidden_states, _ = self.linear_1(caption)
        hidden_states = self.act_1(hidden_states)
        hidden_states, _ = self.linear_2(hidden_states)
        return hidden_states


class LTX2TimestepEmbedder(nn.Module):
    def __init__(self, embedding_dim: int, in_channels: int = 256) -> None:
        super().__init__()
        self.linear_1 = ColumnParallelLinear(
            in_channels, embedding_dim, bias=True, gather_output=True
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Use 'silu' (same as swish) or 'gelu_tanh'
2. Map legacy names in your loader: 'swish'->'silu', 'gelu'->'gelu_tanh' if checkpoint compatibility allows
3. Verify against the config.json shipped with the LTX-2 checkpoint

Example fix

// before
proj = LTX2CaptionProjection(..., act_fn="swish")

// after
proj = LTX2CaptionProjection(..., act_fn="silu")

assert act_fn in ('gelu_tanh', 'silu'), f'unknown act_fn {act_fn!r}'

def is_supported_act_fn(v: str) -> bool:\n    return v in ('gelu_tanh', 'silu')

When it happens

Trigger: Constructing the caption projection with act_fn='gelu' (exact, non-tanh), 'relu', 'swish', or a typo.

Common situations: Config drift between checkpoints (older configs saying 'gelu'); assuming diffusers-style act_fn names ('swish' vs 'silu'); hand-edited configs.

Related errors

• {rope_type=} not supported. Choose between 'interleaved' and
• Modality {modality} is not supported. Supported modalities a
• Hidden size {self.hidden_size} must be divisible by num_atte
• Unsupported qk_norm: {qk_norm}
• LingBotWorld requires cross_attn_norm=True

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0c78f174cb88585c. Report an issue: GitHub.