{"record":{"id":"2c4aabebd0e9b953","repo":"Lightning-AI/pytorch-lightning","slug":"instantiating-your-model-under-the-init-module-c","errorCode":null,"errorMessage":"Instantiating your model under the `init_module` context manager is not supported when used with `BitsandbytesPrecision(..., ignore_modules={self.ignore_modules})` as this may initialize the layers on-device, defeating the purpose of quantization. You can remove `ignore_modules` or remove the `init_module` context manager.","messagePattern":"Instantiating your model under the `init_module` context manager is not supported when used with `BitsandbytesPrecision\\(\\.\\.\\., ignore_modules=(.+?)\\)` as this may initialize the layers on-device, defeating the purpose of quantization\\. You can remove `ignore_modules` or remove the `init_module` context manager\\.","errorType":"validation","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"src/lightning/fabric/plugins/precision/bitsandbytes.py","lineNumber":133,"sourceCode":"            # this will not quantize the model but only replace the layer classes\n            _convert_layers(module, self._linear_cls, self.ignore_modules)\n\n        # set the compute dtype if necessary\n        for m in module.modules():\n            if isinstance(m, bnb.nn.Linear4bit):\n                m.compute_dtype = self.dtype\n                m.compute_type_is_set = False\n        return module\n\n    @override\n    def tensor_init_context(self) -> AbstractContextManager:\n        return _DtypeContextManager(self.dtype)\n\n    @override\n    def module_init_context(self) -> AbstractContextManager:\n        if self.ignore_modules:\n            # cannot patch the Linear class if the user wants to skip some submodules\n            raise RuntimeError(\n                \"Instantiating your model under the `init_module` context manager is not supported when used with\"\n                f\" `BitsandbytesPrecision(..., ignore_modules={self.ignore_modules})` as this\"\n                \" may initialize the layers on-device, defeating the purpose of quantization. You can remove\"\n                \" `ignore_modules` or remove the `init_module` context manager.\"\n            )\n        dtype_ctx = self.tensor_init_context()\n        # TODO: this could also support replacing `Embedding` and `Conv1D`\n        context_manager = _ClassReplacementContextManager({\"torch.nn.Linear\": self._linear_cls})\n        stack = ExitStack()\n        stack.enter_context(dtype_ctx)\n        stack.enter_context(context_manager)\n        return stack\n\n    @override\n    def forward_context(self) -> AbstractContextManager:\n        return _DtypeContextManager(self.dtype)\n\n    @override","sourceCodeStart":115,"sourceCodeEnd":151,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/fabric/plugins/precision/bitsandbytes.py#L115-L151","documentation":"BitsandbytesPrecision quantizes by monkey-patching nn.Linear during module creation under init_module. If ignore_modules is set, the patch cannot be applied selectively, so un-ignored Linears could be initialized on-device unquantized — defeating the purpose. Lightning therefore forbids the combination.","triggerScenarios":"BitsandbytesPrecision(..., ignore_modules={...}) used together with fabric.init_module() context manager when instantiating the model.","commonSituations":"Users excluding some submodules (e.g. lm_head) from quantization while also wanting meta-device/empty init via init_module to save memory before loading weights.","solutions":["Remove ignore_modules and rely on layer-type replacement only (keep init_module)","Drop the init_module context manager (instantiate the model normally) if you must keep ignore_modules","Instantiate the model first, then manually replace/quantize modules, keeping ignore_modules semantics"],"exampleFix":"# before\nplugin = BitsandbytesPrecision(mode=\"nf4\", ignore_modules={\"lm_head\"})\nwith fabric.init_module():\n    model = GPT2(config)  # RuntimeError\n\n# after\nplugin = BitsandbytesPrecision(mode=\"nf4\")\nwith fabric.init_module():\n    model = GPT2(config)","handlingStrategy":"validation","validationCode":"plugin = BitsandbytesPrecision(mode=\"nf4\")\nuse_init_module = not plugin.ignore_modules\nif use_init_module:\n    with fabric.init_module():\n        model = build_model()\nelse:\n    model = build_model()","typeGuard":"def can_use_init_module(plugin) -> bool:\n    return not bool(plugin.ignore_modules)","tryCatchPattern":null,"preventionTips":["Don't combine ignore_modules with init_module","Decide quantization exclusions before choosing the init strategy"],"tags":["bitsandbytes","quantization","init-module","pytorch-lightning"],"backgroundTag":"incompatible-configuration","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}