sgl-project/sglang · error · ValueError

Activation function {act_fn_name!r} is not supported.

Error message

Activation function {act_fn_name!r} is not supported.

What it means

get_act_fn resolves activation functions by lowercased name from the _ACTIVATION_REGISTRY; an unknown name raises ValueError. Supported names are whatever the registry contains (gelu, gelu_tanh, silu, relu, sigmoid, swiglu-style names, etc.).

Source

Thrown at python/sglang/srt/layers/activation.py:478

    "gelu": nn.GELU(),
    "gelu_pytorch_tanh": nn.GELU(approximate="tanh"),
    "gelu_new": NewGELU(),
    "relu2": ReLU2(),
    "xielu": XIELU(),
}


def get_act_fn(
    act_fn_name: str,
    quant_config: Optional[QuantizationConfig] = None,
    intermediate_size: Optional[int] = None,
    input_is_parallel: bool = True,
    params_dtype: Optional[torch.dtype] = None,
) -> nn.Module:
    """Get an activation function by name."""
    act_fn_name = act_fn_name.lower()
    if act_fn_name not in _ACTIVATION_REGISTRY:
        raise ValueError(f"Activation function {act_fn_name!r} is not supported.")

    act_fn = _ACTIVATION_REGISTRY[act_fn_name]
    if quant_config is not None and act_fn_name in quant_config.get_scaled_act_names():
        if intermediate_size is None:
            raise ValueError(
                "intermediate_size must be specified for scaled "
                "activation functions."
            )
        return ScaledActivation(
            act_fn, intermediate_size, input_is_parallel, params_dtype
        )
    return act_fn


def get_cross_encoder_activation_function(config: PretrainedConfig):
    if (
        hasattr(config, "sbert_ce_default_activation_function")
        and config.sbert_ce_default_activation_function is not None

View on GitHub (pinned to 0132848349)

Solutions

1. Check _ACTIVATION_REGISTRY keys (from sglang.srt.layers.activation) for the exact supported names and use the closest match (e.g. 'gelu_new' -> 'gelu')
2. Update SGLang to a version that registers your activation
3. Add the activation to the registry with an implementation and upstream it

Example fix

# before
act = get_act_fn('quick_gelu')
# after
from sglang.srt.layers.activation import _ACTIVATION_REGISTRY
name = 'quick_gelu' if 'quick_gelu' in _ACTIVATION_REGISTRY else 'gelu'
act = get_act_fn(name)

from sglang.srt.layers.activation import _ACTIVATION_REGISTRY
name = hidden_act.lower()
if name not in _ACTIVATION_REGISTRY:
    name = {'gelu_new': 'gelu', 'quick_gelu': 'gelu'}.get(name, 'silu')
act = get_act_fn(name)

def is_supported_act(name: str) -> bool:
    return name.lower() in _ACTIVATION_REGISTRY

try:
    act = get_act_fn(hidden_act)
except ValueError:
    act = get_act_fn('silu')  # safe fallback for registry misses

When it happens

Trigger: Calling get_act_fn('geglu') / 'quick_gelu' / 'silu_fp8' or any string not present in _ACTIVATION_REGISTRY; typo or a name from another framework (vLLM vs SGLang naming differences, e.g. 'gelu_new' vs 'gelu').

Common situations: Model config's hidden_act uses a name SGLang's registry doesn't map (newer model arch, HF-only activation name); porting a model implementation that hardcodes a different activation string.

Related errors

• intermediate_size must be specified for scaled activation fu
• Model config does not contain a _class_name attribute. Only
• Model config does not contain a _class_name attribute. Only
• f"Cannot parse checkpoint quantization for {component_name!r
• f"Transformers-managed {component_name!r} quantization requi

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/71a860f618419cae. Report an issue: GitHub.