{"record":{"id":"2d50b80e4cb83c95","repo":"unslothai/unsloth","slug":"in-features-in-features-is-not-divisible-by-the","errorCode":null,"errorMessage":"in_features {in_features} is not divisible by the ConvRot group {group_size}","messagePattern":"in_features (.+?) is not divisible by the ConvRot group (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":400,"severity":"error","filePath":"studio/backend/core/inference/diffusion_convrot.py","lineNumber":157,"sourceCode":"    features = shape[-1]\n    if features % group_size != 0:\n        raise ValueError(f\"features {features} not divisible by ConvRot group {group_size}\")\n    grouped = x.reshape(-1, features // group_size, group_size)\n    return grouped.matmul(h.to(dtype = x.dtype, device = x.device)).reshape(shape)\n\n\ndef rotate_convrot_weight_(module: Any, group_size: int) -> None:\n    \"\"\"``W <- W @ blockdiag(H).T`` in place, accumulated in float32 and cast back.\n\n    float32 regardless of the stored dtype: each output element becomes a ``group_size``-term dot\n    product, and accumulating that in bfloat16 would spend a visible part of the error budget the\n    rotation exists to save. The offline half only runs once, so the upcast is free.\"\"\"\n    import torch\n\n    weight = module.weight.data\n    out_features, in_features = weight.shape\n    if in_features % group_size:\n        raise ValueError(\n            f\"in_features {in_features} is not divisible by the ConvRot group {group_size}\"\n        )\n    h = build_convrot_hadamard(group_size, device = weight.device, dtype = torch.float32)\n    rotated = torch.matmul(\n        weight.float().reshape(out_features, in_features // group_size, group_size), h.T\n    ).reshape(out_features, in_features)\n    module.weight.data = rotated.to(weight.dtype)\n\n\n@lru_cache(maxsize = None)\ndef convrot_linear_class() -> Any:\n    \"\"\"The ``nn.Linear`` subclass that rotates its input, built lazily so importing this module\n    never imports torch, and built exactly ONCE.\n\n    The cache is not a micro-optimisation, it is the difference between one compiled graph and\n    dozens. A class defined inside a function is a NEW class object on every call, and\n    ``torch.compile`` guards each frame on ``___check_type_id`` of the modules it closes over. So\n    handing every rotated projection its own ConvRotLinear made each one look like a different","sourceCodeStart":139,"sourceCodeEnd":175,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/inference/diffusion_convrot.py#L139-L175","documentation":"rotate_convrot_weight_() must rewrite each weight row as W @ blockdiag(H).T, which requires the Linear's in_features to be an exact multiple of the ConvRot group size. A non-divisible input dimension cannot be block-rotated and the offline rotation aborts before touching the weight.","triggerScenarios":"Calling rotate_convrot_weight_(module, group_size) on an nn.Linear whose weight.shape[1] % group_size != 0; reached from apply_rotation/rotate_weights over an fqn list during checkpoint build.","commonSituations":"Rotating a model whose hidden size is not divisible by the chosen group (e.g. hidden 11008 with group 16 is fine, but 70 x group 64 is not); forgetting to run small-m padding before weight rotation; group size from a different model's config.","solutions":["Pick a power-of-four group that divides in_features (64, 16, or 4 — try the largest that divides)","Pad the Linear's in_features to a multiple of the group (apply_small_m_padding) before the offline rotation","Drop the non-divisible layer from the fqn list; the builder records only the fqns actually rotated"],"exampleFix":"# before\nrotate_convrot_weight_(module, group_size=64)  # module.in_features = 70\n\n# after\ngroup = next(g for g in (64, 16, 4) if module.in_features % g == 0)\nrotate_convrot_weight_(module, group_size=group)","handlingStrategy":"validation","validationCode":"def linear_rotatable(module, group_size: int) -> bool:\n    return isinstance(module, nn.Linear) and module.in_features % group_size == 0","typeGuard":null,"tryCatchPattern":"try:\n    rotate_convrot_weight_(module, group_size)\nexcept ValueError as e:\n    raise BuildError(f\"{fqn}: {e}\") from e  # builder must record only what it actually rotated","preventionTips":["Filter the fqn list up front: only nn.Linear modules with in_features divisible by the group","Run small-m padding before rotation so widths are multiples of the group","Choose the largest power-of-four group that divides every target's in_features"],"tags":["convrot","shape-mismatch","linear-layer","quantization"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}