unslothai/unsloth · error · ValueError

activation rotation target {fqn!r} is already rotated

Error message

activation rotation target {fqn!r} is already rotated

What it means

At load time, a target Linear is already a rotated Linear (is_rotated_linear returned true), meaning the online rotation has been installed twice or the module came pre-rotated. Double rotation would compute x @ H @ H instead of x @ H and silently corrupt outputs, so it is refused.

Source

Thrown at studio/backend/core/inference/diffusion_convrot.py:362

        raise ValueError(problem)

    from torch import nn

    group_size = int(metadata[ROTATION_GROUP_KEY])
    fqns = list(metadata[ROTATION_FQNS_KEY])
    modules = dict(transformer.named_modules())
    missing = [fqn for fqn in fqns if fqn not in modules]
    if missing:
        raise ValueError(
            f"activation rotation names {len(missing)} fqn(s) this model does not have "
            f"(e.g. {missing[0]!r}); the checkpoint and this build disagree about the model"
        )
    for fqn in fqns:
        module = modules[fqn]
        if not isinstance(module, nn.Linear):
            raise ValueError(f"activation rotation target {fqn!r} is not an nn.Linear")
        if is_rotated_linear(module):
            raise ValueError(f"activation rotation target {fqn!r} is already rotated")
        if module.in_features % group_size:
            raise ValueError(
                f"activation rotation target {fqn!r} has in_features {module.in_features}, "
                f"which the recorded group {group_size} does not divide"
            )
    # Every target is validated before ANY is swapped. A partial install is the one outcome worse
    # than either end state: the rotated half still renders, just wrongly, so there is nothing to
    # notice and nothing to fall back from.
    for fqn in fqns:
        _install_rotation(modules[fqn], group_size)
    try:
        setattr(
            transformer,
            CONVROT_ATTR,
            {"kind": CONVROT_KIND, "group": group_size, "linears": len(fqns)},
        )
    except Exception:  # noqa: BLE001 -- the marker is a diagnostic, never the mechanism
        pass

View on GitHub (pinned to 203007d190)

Solutions

1. Apply the rotation exactly once per model instance: guard with is_rotated_linear(module) before calling
2. Rebuild the transformer from scratch (fresh from_pretrained/config) before retrying a failed rotated load
3. Audit for duplicate call sites of apply_activation_rotation in the load path

Example fix

# before
apply_activation_rotation(transformer, metadata)  # called again on retry

# after
if any(is_rotated_linear(m) for m in transformer.modules()):
    transformer = rebuild_fresh_model(config)  # or skip re-applying
apply_activation_rotation(transformer, metadata)

from studio.backend.core.inference.diffusion_convrot import is_rotated_linear

def needs_rotation(transformer, fqns) -> bool:
    mods = dict(transformer.named_modules())
    return any(not is_rotated_linear(mods[fqn]) for fqn in fqns)

from studio.backend.core.inference.diffusion_convrot import is_rotated_linear

def is_rotated(module) -> bool:
    """True if the online rotation is already installed on this module."""
    return is_rotated_linear(module)

try:
    apply_activation_rotation(transformer, metadata)
except ValueError as e:
    if "already rotated" in str(e):
        rebuild_model_and_retry()  # fresh instance, then apply once
    raise

When it happens

Trigger: Calling apply_activation_rotation twice on the same transformer (e.g. a retry loop or a second load path re-applying metadata); or loading weights into a transformer that already had _install_rotation run — including the meta-device retry path that re-applies after a rebuild.

Common situations: Retrying a failed load without rebuilding the model; a code change that moved apply_activation_rotation into a helper invoked from two places; checkpoint contains already-rotated modules plus metadata asking to rotate again.

Related errors

• ConvRot group size must be a power of 4, got {size}
• features {features} not divisible by ConvRot group {group_si
• in_features {in_features} is not divisible by the ConvRot gr
• ConvRot group size must be a power of 4, got {group_size!r}
• cannot rotate {fqn!r}: not an nn.Linear on this model

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/920daa6ce87ca1ad. Report an issue: GitHub.