unslothai/unsloth · error · ValueError

activation rotation target {fqn!r} is not an nn.Linear

Error message

activation rotation target {fqn!r} is not an nn.Linear

What it means

At load time, a fqn recorded in the checkpoint's rotation metadata resolves to a module that is not an nn.Linear in this model. Only plain Linears can host the online rotation, and every target is validated before any swap so a partial install cannot happen.

Source

Thrown at studio/backend/core/inference/diffusion_convrot.py:360

    problem = rotation_metadata_error(metadata)
    if problem:
        raise ValueError(problem)

    from torch import nn

    group_size = int(metadata[ROTATION_GROUP_KEY])
    fqns = list(metadata[ROTATION_FQNS_KEY])
    modules = dict(transformer.named_modules())
    missing = [fqn for fqn in fqns if fqn not in modules]
    if missing:
        raise ValueError(
            f"activation rotation names {len(missing)} fqn(s) this model does not have "
            f"(e.g. {missing[0]!r}); the checkpoint and this build disagree about the model"
        )
    for fqn in fqns:
        module = modules[fqn]
        if not isinstance(module, nn.Linear):
            raise ValueError(f"activation rotation target {fqn!r} is not an nn.Linear")
        if is_rotated_linear(module):
            raise ValueError(f"activation rotation target {fqn!r} is already rotated")
        if module.in_features % group_size:
            raise ValueError(
                f"activation rotation target {fqn!r} has in_features {module.in_features}, "
                f"which the recorded group {group_size} does not divide"
            )
    # Every target is validated before ANY is swapped. A partial install is the one outcome worse
    # than either end state: the rotated half still renders, just wrongly, so there is nothing to
    # notice and nothing to fall back from.
    for fqn in fqns:
        _install_rotation(modules[fqn], group_size)
    try:
        setattr(
            transformer,
            CONVROT_ATTR,
            {"kind": CONVROT_KIND, "group": group_size, "linears": len(fqns)},
        )

View on GitHub (pinned to 203007d190)

Solutions

1. Call apply_activation_rotation after load_state_dict but BEFORE apply_small_m_padding, as documented
2. Ensure base model and rotation metadata come from the same architecture revision
3. Fall back to the dense checkpoint if the build genuinely differs

Example fix

# before
apply_small_m_padding(transformer, 64)
apply_activation_rotation(transformer, metadata)  # fqns now name wrappers

# after
apply_activation_rotation(transformer, metadata)  # immediately after load_state_dict
apply_small_m_padding(transformer, 64)

from torch import nn

def rotation_targets_ok(transformer, fqns) -> bool:
    mods = dict(transformer.named_modules())
    return all(fqn in mods and isinstance(mods[fqn], nn.Linear) for fqn in fqns)

from torch import nn

def is_rotation_target(module) -> bool:
    return isinstance(module, nn.Linear)  # wrappers created by padding fail this by design

try:
    apply_activation_rotation(transformer, metadata)
except ValueError as e:
    raise CheckpointRefused(str(e)) from e  # loader converts to dense fallback

When it happens

Trigger: apply_activation_rotation on a model where a recorded fqn now names an Embedding, LayerNorm, fused module, or Linear subclass wrapper that fails isinstance(module, nn.Linear); typical when padding (apply_small_m_padding) or another transform reparented modules first, or the architecture changed.

Common situations: Calling apply_activation_rotation AFTER apply_small_m_padding (the docstring requires the reverse order, because padding reparents Linears under a wrapper); architecture revision replacing a Linear with a fused implementation; loading the meta-retry path that rebuilt modules differently.

Related errors

• cannot rotate {fqn!r}: not an nn.Linear on this model
• activation rotation names {len(missing)} fqn(s) this model d
• ConvRot group size must be a power of 4, got {size}
• features {features} not divisible by ConvRot group {group_si
• in_features {in_features} is not divisible by the ConvRot gr

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/b001fac9dff5078b. Report an issue: GitHub.