sgl-project/sglang · error · ValueError

intermediate_size must be specified for scaled activation fu

Error message

intermediate_size must be specified for scaled activation functions.

What it means

When a quant_config marks an activation as scaled (e.g. fp8 ScaledActivation), get_act_fn needs intermediate_size to allocate the per-channel scales tensor. Passing quant_config with a scaled activation but intermediate_size=None raises ValueError.

Source

Thrown at python/sglang/srt/layers/activation.py:483

}


def get_act_fn(
    act_fn_name: str,
    quant_config: Optional[QuantizationConfig] = None,
    intermediate_size: Optional[int] = None,
    input_is_parallel: bool = True,
    params_dtype: Optional[torch.dtype] = None,
) -> nn.Module:
    """Get an activation function by name."""
    act_fn_name = act_fn_name.lower()
    if act_fn_name not in _ACTIVATION_REGISTRY:
        raise ValueError(f"Activation function {act_fn_name!r} is not supported.")

    act_fn = _ACTIVATION_REGISTRY[act_fn_name]
    if quant_config is not None and act_fn_name in quant_config.get_scaled_act_names():
        if intermediate_size is None:
            raise ValueError(
                "intermediate_size must be specified for scaled "
                "activation functions."
            )
        return ScaledActivation(
            act_fn, intermediate_size, input_is_parallel, params_dtype
        )
    return act_fn


def get_cross_encoder_activation_function(config: PretrainedConfig):
    if (
        hasattr(config, "sbert_ce_default_activation_function")
        and config.sbert_ce_default_activation_function is not None
    ):

        function_name = config.sbert_ce_default_activation_function
        assert function_name.startswith("torch.nn.modules."), (
            "Loading of activation functions is restricted to "

View on GitHub (pinned to 0132848349)

Solutions

Pass intermediate_size explicitly to get_act_fn wherever quant_config is non-None
If the activation should not be scaled, fix quant_config.get_scaled_act_names()/config so the name is excluded
Update the model implementation to plumb intermediate_size from its constructor into get_act_fn

Example fix

# before
self.act_fn = get_act_fn(hidden_act, quant_config=self.quant_config)
# after
self.act_fn = get_act_fn(
    hidden_act,
    quant_config=self.quant_config,
    intermediate_size=self.intermediate_size_per_partition,
    input_is_parallel=True,
)

Defensive patterns

Strategy: validation

Validate before calling

from sglang.srt.layers.activation import get_act_fn
scaled = quant_config is not None and hidden_act in quant_config.get_scaled_act_names()
if scaled and intermediate_size is None:
    intermediate_size = config.intermediate_size
act = get_act_fn(hidden_act, quant_config=quant_config, intermediate_size=intermediate_size)

Type guard

def needs_intermediate_size(name: str, quant_config) -> bool:
    return quant_config is not None and name in quant_config.get_scaled_act_names()

Prevention

Always pass intermediate_size when quant_config is not None
Audit custom MLP/MoE code to forward intermediate_size into get_act_fn

When it happens

Trigger: Calling get_act_fn('gelu', quant_config=fp8_config) without intermediate_size, where fp8_config.get_scaled_act_names() includes 'gelu'. Typical when an MLP/GPU layer built before quantization config plumbing passed intermediate_size.

Common situations: Loading an fp8/int8 quantized checkpoint whose act scales are stored, with model code path (e.g. a custom MoE MLP) that forgot to forward intermediate_size into get_act_fn.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/de21ad61a3e6cb6d. Report an issue: GitHub.