unslothai/unsloth · error · ValueError

activation rotation target {fqn!r} has in_features {module.i

Error message

activation rotation target {fqn!r} has in_features {module.in_features}, which the recorded group {group_size} does not divide

What it means

At load time, a recorded target Linear's in_features is not divisible by the group size recorded in checkpoint metadata. The online rotation reshapes the last dimension into blocks of group_size, so a non-divisible feature count is structurally impossible to rotate and is refused before any swap (all targets are validated first).

Source

Thrown at studio/backend/core/inference/diffusion_convrot.py:364

    from torch import nn

    group_size = int(metadata[ROTATION_GROUP_KEY])
    fqns = list(metadata[ROTATION_FQNS_KEY])
    modules = dict(transformer.named_modules())
    missing = [fqn for fqn in fqns if fqn not in modules]
    if missing:
        raise ValueError(
            f"activation rotation names {len(missing)} fqn(s) this model does not have "
            f"(e.g. {missing[0]!r}); the checkpoint and this build disagree about the model"
        )
    for fqn in fqns:
        module = modules[fqn]
        if not isinstance(module, nn.Linear):
            raise ValueError(f"activation rotation target {fqn!r} is not an nn.Linear")
        if is_rotated_linear(module):
            raise ValueError(f"activation rotation target {fqn!r} is already rotated")
        if module.in_features % group_size:
            raise ValueError(
                f"activation rotation target {fqn!r} has in_features {module.in_features}, "
                f"which the recorded group {group_size} does not divide"
            )
    # Every target is validated before ANY is swapped. A partial install is the one outcome worse
    # than either end state: the rotated half still renders, just wrongly, so there is nothing to
    # notice and nothing to fall back from.
    for fqn in fqns:
        _install_rotation(modules[fqn], group_size)
    try:
        setattr(
            transformer,
            CONVROT_ATTR,
            {"kind": CONVROT_KIND, "group": group_size, "linears": len(fqns)},
        )
    except Exception:  # noqa: BLE001 -- the marker is a diagnostic, never the mechanism
        pass
    if logger is not None:
        logger.info(

View on GitHub (pinned to 203007d190)

Solutions

1. Load the exact base model the checkpoint was built from (same hidden sizes)
2. Re-run the offline rotation with a group that divides every target's in_features
3. Ensure apply_small_m_padding ran during the build so widths are multiples of the group, and that the loader reproduces the same padding order

Example fix

# before
apply_activation_rotation(transformer, metadata)  # group 64, layer in_features=70

# after
# rebuild checkpoint with a group that divides every target:
apply_rotation(transformer, fqns, group_size=next(g for g in (64,16,4) if all(m.in_features % g == 0 for m in targets)))

def groups_fit(transformer, fqns, group_size: int) -> bool:
    mods = dict(transformer.named_modules())
    return all(mods[fqn].in_features % group_size == 0 for fqn in fqns)

try:
    apply_activation_rotation(transformer, metadata)
except ValueError as e:
    return dense_fallback(str(e))  # checkpoint widths and model disagree; refuse the checkpoint

When it happens

Trigger: Loading rotated checkpoint metadata (group_size G) onto a model whose Linear widths are not multiples of G — e.g. group 64 recorded but this revision of the model has a projection with 70/96/11008-vs-group mismatch; or small-m padding was skipped before the offline build.

Common situations: Architecture revision changed hidden sizes after the checkpoint was rotated; checkpoint built with padding applied but loaded against unpadded weights; group size tuned on one model and reused on another.

Related errors

• features {features} not divisible by ConvRot group {group_si
• in_features {in_features} is not divisible by the ConvRot gr
• unsupported activation rotation {kind!r} (this build impleme
• activation rotation names {len(missing)} fqn(s) this model d
• ConvRot group size must be a power of 4, got {size}

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/eb3a9030e1b15bf8. Report an issue: GitHub.