sgl-project/sglang · error · ValueError

norm_type must be one of "layer" and "rms"

Error message

norm_type must be one of "layer" and "rms"

What it means

fused_norm_scale_shift dispatches on a norm_type string and only 'layer' (LayerNorm) and 'rms' (RMSNorm) have kernel implementations. Any other string reaches the else branch and raises.

Source

Thrown at python/sglang/kernels/ops/diffusion/norm/scale_residual_norm_cutedsl.py:314

        weight = 1 if weight is None else weight
        bias = 0 if bias is None else bias
        ResOut, Residual, Gate = 0, 0, 1
        torch_tensors = [y, ResOut, Residual, x, Gate, weight, bias, scale, shift]
        # Compile cache
        hash_key = ScaleResidualNormScaleShift.make_hash_key(norm_type, *torch_tensors)
        compiled_fn = _COMPILE_CACHE.get(hash_key)
        if compiled_fn is None:
            kernel = ScaleResidualNormScaleShift(D, norm_type)
            fake_sig_args = [to_fake_cute_args(t) for t in torch_tensors]
            compiled_fn = cute.compile(
                kernel, *fake_sig_args, options="--enable-tvm-ffi"
            )
            _COMPILE_CACHE[hash_key] = compiled_fn
        # Execute
        compiled_fn(*torch_tensors, eps, stream)
        return y
    else:
        raise ValueError('norm_type must be one of "layer" and "rms"')


@fused_norm_scale_shift.register_fake
def _fused_norm_scale_shift_fake(x, weight, bias, scale, shift, norm_type, eps=1e-5):
    y = x.new_empty(x.shape)
    return y


@torch.library.custom_op(
    "sglang::fused_scale_residual_norm_scale_shift", mutates_args=()
)
def fused_scale_residual_norm_scale_shift(
    residual: torch.Tensor,
    x: torch.Tensor,
    gate: Optional[torch.Tensor],  # Union[Optional[torch.Tensor], int] indeed
    weight: Optional[torch.Tensor],
    bias: Optional[torch.Tensor],
    scale: torch.Tensor,

View on GitHub (pinned to 0132848349)

Solutions

1. Pass exactly 'layer' or 'rms'
2. Normalize/alias config strings before the call (e.g. map 'layernorm'->'layer')
3. Add a new branch + kernel only if you actually extended the kernel file

Example fix

# before
fused_norm_scale_shift(x, w, b, s, sh, cfg.norm_type)  # 'LayerNorm'
# after
nt = cfg.norm_type.lower().replace('norm', '')  # -> 'layer' / 'rms'
fused_norm_scale_shift(x, w, b, s, sh, nt)

assert norm_type in ("layer", "rms"), f"bad norm_type {norm_type!r}"

def is_valid_norm_type(nt: str) -> bool:
    return nt in ("layer", "rms")

When it happens

Trigger: Calling fused_norm_scale_shift(x, ..., norm_type) with norm_type not exactly 'layer' or 'rms' — e.g. 'layernorm', 'RMS', 'group', or a typo.

Common situations: Config-driven norm names ('LayerNorm' capitalized) passed through without normalization; new norm variants wired to this entry point.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• unknown norm_type {norm_type}
• {path}.kind unsupported for ref2va: {kind!r}
• Invalid deepep_mode: {self.deepep_mode}
• v_cache must be provided
• q can only be None when only_qv=True

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f8bd2ccf5db0bac7. Report an issue: GitHub.