Lightning-AI/pytorch-lightning · error · TypeError

You are using the bitsandbytes precision plugin, but your mo

Error message

You are using the bitsandbytes precision plugin, but your model has no Linear layers. This plugin won't work for your model.

What it means

BitsandbytesPrecision only quantizes torch.nn.Linear layers. convert_module checks the model contains at least one Linear; if not, it raises TypeError to make clear that the plugin would silently do nothing (no quantization would happen).

Source

Thrown at src/lightning/fabric/plugins/precision/bitsandbytes.py:107

        globals_ = globals()
        mode_to_cls = {
            "nf4": globals_["_NF4Linear"],
            "nf4-dq": globals_["_NF4DQLinear"],
            "fp4": globals_["_FP4Linear"],
            "fp4-dq": globals_["_FP4DQLinear"],
            "int8-training": globals_["_Linear8bitLt"],
            "int8": globals_["_Int8LinearInference"],
        }
        self._linear_cls = mode_to_cls[mode]
        self.dtype = dtype
        self.ignore_modules = ignore_modules or set()

    @override
    def convert_module(self, module: torch.nn.Module) -> torch.nn.Module:
        # avoid naive users thinking they quantized their model
        if not any(isinstance(m, torch.nn.Linear) for m in module.modules()):
            raise TypeError(
                "You are using the bitsandbytes precision plugin, but your model has no Linear layers. This plugin"
                " won't work for your model."
            )

        # convert modules if they haven't been converted already
        bnb = _import_bitsandbytes()
        if not any(isinstance(m, (bnb.nn.Linear8bitLt, bnb.nn.Linear4bit)) for m in module.modules()):
            # this will not quantize the model but only replace the layer classes
            _convert_layers(module, self._linear_cls, self.ignore_modules)

        # set the compute dtype if necessary
        for m in module.modules():
            if isinstance(m, bnb.nn.Linear4bit):
                m.compute_dtype = self.dtype
                m.compute_type_is_set = False
        return module

    @override

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Verify the model actually uses nn.Linear layers (sum(1 for m in model.modules() if isinstance(m, nn.Linear)))
2. Rewrite matmul-based layers as nn.Linear so they can be replaced by bnb Linear8bitLt/Linear4bit
3. Don't use the bitsandbytes plugin for models without Linear layers

Example fix

# before
# model uses self.w @ x instead of nn.Linear
fabric = Fabric(plugins=BitsandbytesPrecision(mode="nf4"))
model = fabric.setup(model)  # TypeError

# after
class Block(nn.Module):
    def __init__(self):
        self.proj = nn.Linear(d, d)  # uses nn.Linear
    def forward(self, x):
        return self.proj(x)

import torch

def model_has_linear(module: torch.nn.Module) -> bool:
    return any(isinstance(m, torch.nn.Linear) for m in module.modules())

assert model_has_linear(model), "bitsandbytes plugin requires nn.Linear layers"

def model_has_linear(module: torch.nn.Module) -> bool:
    return any(isinstance(m, torch.nn.Linear) for m in module.modules())

When it happens

Trigger: Calling setup/fabric.setup(module) with BitsandbytesPrecision on a model with no nn.Linear layers (e.g. pure Conv/Transformer-with-conv/MLP built from conv1d, embeddings only).

Common situations: Applying the bnb plugin to CNNs, embedding-only models, or models whose linear ops are implemented via einsum/matmul on parameters instead of nn.Linear modules.

Related errors

• {mode!r} only works with `dtype=torch.float16`, but you chos
• Instantiating your model under the `init_module` context man
• str(_BITSANDBYTES_AVAILABLE)
• Bitsandbytes is only supported on CUDA GPUs.
• Device should be CPU, got {device} instead.

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/4252767194cf609d. Report an issue: GitHub.