hiyouga/LlamaFactory · error · RuntimeError

{self.class.name} has no RMSNorm weight for NPU RMSN

Error message

{self.__class__.__name__} has no RMSNorm weight for NPU RMSNorm kernel.

What it means

The patched NPU RMSNorm forward reads the module's `weight` attribute; if the module has no weight (weight is None or the attribute is missing), the NPU op has no scale tensor and the code raises RuntimeError naming the module class before calling torch_npu.npu_rms_norm.

Source

Thrown at src/llamafactory/v1/plugins/model_plugins/kernels/ops/rms_norm/npu_rms_norm.py:58

else:
    _TORCH_NPU_IMPORT_ERROR = None


def npu_rms_norm_forward(self, hidden_states):
    """NPU forward implementation for standard RMSNorm.

    Args:
        self (nn.Module): The RMSNorm module instance with ``weight`` and either ``variance_epsilon`` or ``eps``.
        hidden_states (Tensor): Input hidden states tensor.

    Returns:
        Tensor: Normalized tensor consistent with the baseline RMSNorm behavior.
    """
    _eps = getattr(self, "variance_epsilon", None) or getattr(self, "eps", 1e-6)

    weight = getattr(self, "weight", None)
    if weight is None:
        raise RuntimeError(f"{self.__class__.__name__} has no RMSNorm weight for NPU RMSNorm kernel.")

    effective_weight = weight.float()

    return torch_npu.npu_rms_norm(hidden_states, effective_weight.to(hidden_states.dtype), epsilon=_eps)[0]


def npu_residual_rms_norm_forward(self, hidden_states):
    """NPU forward implementation for residual RMSNorm.

    Residual RMSNorm uses ``scale = 1.0 + weight`` where ``weight`` is initialized
    to 0 in the original transformers implementation.

    Args:
        self (nn.Module): The residual RMSNorm module with ``weight`` and either ``variance_epsilon`` or ``eps``.
        hidden_states (Tensor): Input hidden states tensor.

    Returns:
        Tensor: Normalized tensor consistent with residual RMSNorm behavior.

View on GitHub (pinned to f28afaf635)

Solutions

Inspect the failing module class from the error message and confirm its weight attribute exists and is a Parameter
If the norm legitimately has no affine weight, exclude that module/model type from npu_fused_rmsnorm patches
Verify the checkpoint loads fully (no missing-keys warnings for norm weights)
Align transformers version with what _MODEL_TYPE_TO_PATCHES was built against

Defensive patterns

Strategy: try-catch

Validate before calling

for name, m in model.named_modules():
    if isinstance(m, nn.RMSNorm) and getattr(m, "weight", None) is None:
        raise ValueError(f"{name} has no RMSNorm weight; cannot use npu_fused_rmsnorm")

Try / catch

try:
    patched = apply_npu_rmsnorm(model)
except RuntimeError as e:
    if "no RMSNorm weight" in str(e):
        # module without affine weight: skip patching this module
        log.warning("skip NPU RMSNorm: %s", e)
    else:
        raise

Prevention

Confirm checkpoints load all norm weights (watch missing-keys warnings)
Exclude weightless norm variants from the patch map

When it happens

Trigger: A module matched by _MODEL_TYPE_TO_PATCHES whose `weight` is None — e.g. an RMSNorm constructed with elementwise_affine=False, a partially initialized/tied-weight module, or a custom norm class that stores the scale under a different attribute name.

Common situations: Loading checkpoints where the norm weight was not restored (remapped state dict); custom model classes with renamed attributes; newer transformers refactors moving the weight off the expected attribute.

Related errors

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/dc9f14b020aeec1b. Report an issue: GitHub.