hiyouga/LlamaFactory · error · ValueError

{self.class.name} requires a gate tensor for NPU Gat

Error message

{self.__class__.__name__} requires a gate tensor for NPU Gated RMSNorm.

What it means

The gated RMSNorm NPU forward applies RMSNorm then multiplies by SiLU(gate); the gate tensor is mandatory. If forward is called with gate=None (the patched signature allows it), the code raises ValueError naming the module class, rather than passing None into a fused op.

Source

Thrown at src/llamafactory/v1/plugins/model_plugins/kernels/ops/rms_norm/npu_rms_norm.py:108

    """NPU forward implementation for Gated RMSNorm with high-precision FP32 computation.

    This function performs RMSNorm and gated SiLU multiplication in FP32 for numerical
    stability. The supported gated RMSNorm modules use ``scale = weight`` with weight
    initialized to 1, unlike the residual RMSNorm variants that use ``1.0 + weight``.

    Args:
        self (nn.Module): The Gated RMSNorm module instance.
        hidden_states (Tensor): Input hidden states tensor.
        gate (Tensor): Gate tensor for SiLU activation.

    Returns:
        Tensor: Output tensor cast back to the original input dtype.

    Raises:
        ValueError: If the gate tensor is not provided.
    """
    if gate is None:
        raise ValueError(f"{self.__class__.__name__} requires a gate tensor for NPU Gated RMSNorm.")

    input_dtype = hidden_states.dtype
    hidden_states = hidden_states.to(torch.float32)
    _eps = getattr(self, "variance_epsilon", None) or getattr(self, "eps", 1e-6)

    hidden_states = torch_npu.npu_rms_norm(hidden_states, self.weight.float(), epsilon=_eps)[0]
    hidden_states = hidden_states * F.silu(gate.to(torch.float32))

    return hidden_states.to(input_dtype)


_MODEL_TYPE_TO_PATCHES = {
    "qwen3": {
        "Qwen3RMSNorm": npu_rms_norm_forward,
    },
    "qwen3_moe": {
        "Qwen3MoeRMSNorm": npu_rms_norm_forward,
    },

View on GitHub (pinned to f28afaf635)

Solutions

Ensure every call site of the gated RMSNorm module passes the gate tensor
If the layer genuinely has no gate, prevent that module from being patched (check _MODEL_TYPE_TO_PATCHES matching for the model type)
Upgrade/downgrade transformers so gated and non-gated variants use distinct classes matching the patch map

Defensive patterns

Strategy: validation

Validate before calling

# before invoking the patched module, ensure a gate tensor is available
if gate is None:
    raise ValueError("gated RMSNorm requires gate; this layer must not be patched")

Try / catch

try:
    out = module(hidden_states, gate=gate)
except ValueError as e:
    if "requires a gate tensor" in str(e):
        # fall back to unpatched forward
        out = module._original_forward(hidden_states, gate)
    else:
        raise

Prevention

Ensure all call sites pass gate for gated-norm models
Keep patched/unpatched classes distinct when customizing model code

When it happens

Trigger: Calling a patched gated RMSNorm module's forward without the gate argument — a caller/site written for the unpatched module where gate was optional, or a model path (e.g. non-gated layer variant) that never produces a gate tensor while the module class got patched.

Common situations: Model variants sharing a module class where only some paths supply a gate; upstream code calling hidden_states-only forward after the NPU patch was applied; partial application of patches across mixed layers.

Related errors

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/724f249e150e13bf. Report an issue: GitHub.