{"record":{"id":"555694c8b2e8b07e","repo":"unslothai/unsloth","slug":"fqn-refusing-to-pad-a-quantized-linear-whose-ac","errorCode":null,"errorMessage":"{fqn}: refusing to pad a quantized Linear whose activation granularity is not provably per row. Padding replicates row 0, which is exact only when each kept row's scale is computed from that row alone; under a calibrated or per-tensor activation scale it would silently change every output.","messagePattern":"(.+?): refusing to pad a quantized Linear whose activation granularity is not provably per row\\. Padding replicates row 0, which is exact only when each kept row's scale is computed from that row alone; under a calibrated or per-tensor activation scale it would silently change every output\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/inference/diffusion_quant_pad.py","lineNumber":281,"sourceCode":"    row raises ``RuntimeError``: see the module docstring for why silence is the wrong answer.\n    \"\"\"\n    done: list[str] = []\n    for fqn in sorted(set(fqns)):\n        parent_name, _, leaf = fqn.rpartition(\".\")\n        try:\n            parent = model.get_submodule(parent_name) if parent_name else model\n            module = getattr(parent, leaf)\n        except AttributeError:\n            # A family token that matches nothing on this checkpoint variant is not an error:\n            # the pruned and dense H3 trees differ, and callers pass a name list, not a promise.\n            continue\n        # Skips a dense Linear (``F.linear`` has no row floor to clear, and there is no\n        # granularity to prove) and, by the same gate, an already-wrapped one: ``PadToMinM`` is\n        # not an ``nn.Linear``, so re-wrapping cannot nest the padding and double the row count.\n        if not is_quantized_linear(module):\n            continue\n        if require_per_row and activation_granularity_is_per_row(module) is not True:\n            raise RuntimeError(\n                f\"{fqn}: refusing to pad a quantized Linear whose activation granularity is not \"\n                f\"provably per row. Padding replicates row 0, which is exact only when each \"\n                f\"kept row's scale is computed from that row alone; under a calibrated or \"\n                f\"per-tensor activation scale it would silently change every output.\"\n            )\n        setattr(parent, leaf, PadToMinM(module, min_m = min_m, pad_to = pad_to))\n        done.append(fqn)\n    return tuple(done)\n\n\ndef matching_linear_fqns(model: nn.Module, name_tokens: Iterable[str]) -> tuple[str, ...]:\n    \"\"\"Every quantized-Linear fqn in ``model`` containing one of ``name_tokens`` (substring,\n    case-insensitive) -- the same matching rule ``make_filter_fn`` uses for exclusions, so the\n    pad list and the exclude list are read the same way.\"\"\"\n    tokens = tuple(t.lower() for t in name_tokens if t)\n    if not tokens:\n        return ()\n    return tuple(","sourceCodeStart":263,"sourceCodeEnd":299,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/inference/diffusion_quant_pad.py#L263-L299","documentation":"The quant-pad installer refuses to wrap a quantized nn.Linear in PadToMinM when require_per_row is set and activation_granularity_is_per_row(module) is not provably True. Padding replicates row 0, which is numerically exact only if each row's activation scale was computed per row; under calibrated or per-tensor activation scales, replication would silently corrupt every output, so it is a hard RuntimeError.","triggerScenarios":"Calling the pad-install routine (require_per_row=True) on a model whose quantized Linears carry per-tensor or calibrated activation scales instead of per-row — e.g. a checkpoint quantized with a calibration dataset, or a torchao config that used per-tensor input scales.","commonSituations":"Mixing checkpoints quantized under a different scheme than the padding code assumes; upgrading a quantization tool that changed default activation granularity; hand-modified quant configs.","solutions":["Use a checkpoint whose activation quantization granularity is per-row (standard dynamic per-row scales)","Re-quantize the model without calibration / with per-row activation scales","Skip padding for these Linears (drop their names from the pad target list) if row-count alignment is not needed for them"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"# Before installing pads, verify granularity on each target\nfqns = matching_linear_fqns(model, tokens)\nfor fqn in fqns:\n    mod = model.get_submodule(fqn)\n    if require_per_row and activation_granularity_is_per_row(mod) is not True:\n        skip_or_fail(fqn)  # exclude from the pad list instead of raising","typeGuard":"def safe_to_pad(module) -> bool:\n    return is_quantized_linear(module) and activation_granularity_is_per_row(module) is True","tryCatchPattern":"try:\n    install_pads(model, targets, require_per_row=True)\nexcept RuntimeError as e:\n    if \"refusing to pad\" in str(e):\n        log_and_exclude(e)  # record the fqn, continue without padding it","preventionTips":["Only apply PadToMinM to dynamically per-row-quantized Linears","Keep calibrated-quant checkpoints out of padded code paths","Treat the refusal as a numerics guard, never catch-and-force"],"tags":["quantization","numerics","validation","padding"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}