{"record":{"id":"b407326bd8537f5b","repo":"unslothai/unsloth","slug":"convrot-group-size-must-be-a-power-of-4-got-size","errorCode":null,"errorMessage":"ConvRot group size must be a power of 4, got {size}","messagePattern":"ConvRot group size must be a power of 4, got (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":400,"severity":"error","filePath":"studio/backend/core/inference/diffusion_convrot.py","lineNumber":120,"sourceCode":"    device: Any = \"cpu\",\n    dtype: Any = None,\n) -> Any:\n    \"\"\"The normalized regular Hadamard matrix ConvRot rotates by. Cached per (size, device, dtype).\n\n    Built as ``kron(H4, H4, ...) / sqrt(size)``, which is both symmetric and orthogonal -- the\n    property the offline/online pair relies on, since it means the same matrix undoes itself and\n    the weight side can use ``H.T`` interchangeably with ``H``. Building directly in ``dtype`` is\n    exact for every float type: the entries are +-1 and the normalizer is a power of two.\"\"\"\n    import torch\n\n    if dtype is None:\n        dtype = torch.float32\n    key = (size, str(device), dtype)\n    cached = _HADAMARD_CACHE.get(key)\n    if cached is not None:\n        return cached\n    if not is_power_of_four(size):\n        raise ValueError(f\"ConvRot group size must be a power of 4, got {size}\")\n    h4 = torch.tensor(\n        [[1, 1, 1, -1], [1, 1, -1, 1], [1, -1, 1, 1], [-1, 1, 1, 1]],\n        dtype = dtype,\n        device = device,\n    )\n    h = h4\n    current = 4\n    while current < size:\n        h = torch.kron(h, h4)\n        current *= 4\n    h = h / (size**0.5)\n    _HADAMARD_CACHE[key] = h\n    return h\n\n\ndef rotate_convrot_activation(x: Any, h: Any, group_size: int) -> Any:\n    \"\"\"``x @ H`` blockwise over the last dimension.\"\"\"\n    shape = x.shape","sourceCodeStart":102,"sourceCodeEnd":138,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/inference/diffusion_convrot.py#L102-L138","documentation":"build_convrot_hadamard() constructs a scaled Hadamard matrix via repeated Kronecker products of a 4x4 base, which is only defined for sizes 4, 16, 64, ... (powers of four). Any other size raises before any tensor work.","triggerScenarios":"Calling build_convrot_hadamard(size) with size not a power of four (e.g. 8, 32, 48, 0); usually reached from rotate_convrot_weight_/activation-rotation code paths that pass an unvalidated group_size.","commonSituations":"Copy-pasting a SpinQuant/Quarot-style config that uses group 128 (power of two, not four); tweaking DEFAULT_CONVROT_GROUPSIZE to 8 or 32; computing group size from a formula (in_features/32) that lands off the power-of-four ladder.","solutions":["Use a power of four: 4, 16, 64 (the Kronecker construction cannot build anything else)","Validate group sizes at config load with is_power_of_four() so the mistake is caught at startup, not mid-inference","If a smaller/larger rotation is needed, pick the nearest power of four and pad/choose divisible in_features accordingly"],"exampleFix":"# before\nh = build_convrot_hadamard(32)   # ValueError: power of two, not four\n\n# after\nh = build_convrot_hadamard(64)    # 4, 16, 64, ... only","handlingStrategy":"validation","validationCode":"from studio.backend.core.inference.diffusion_convrot import is_power_of_four\n\ndef hadamard_size_valid(size: int) -> bool:\n    return is_power_of_four(size)  # 4, 16, 64, ...","typeGuard":null,"tryCatchPattern":"try:\n    h = build_convrot_hadamard(group_size, device=dev, dtype=dt)\nexcept ValueError as e:\n    raise ConfigError(str(e)) from e  # fail the build immediately, never fall back to a wrong matrix","preventionTips":["Validate group sizes once at config load, not per layer","Remember powers of FOUR (4/16/64), not the powers of two (128/256) used by other Hadamard schemes","Treat group size as a build-time constant recorded into checkpoint metadata; never derive it per request"],"tags":["convrot","hadamard","input-validation","quantization"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}