{"record":{"id":"654b2a4cdc7ec98a","repo":"invoke-ai/InvokeAI","slug":"the-base-model-quantization-format-likely-bitsand","errorCode":null,"errorMessage":"The base model quantization format (likely bitsandbytes) is not compatible with DoRA patches.","messagePattern":"The base model quantization format \\(likely bitsandbytes\\) is not compatible with DoRA patches\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"invokeai/backend/patches/layers/dora_layer.py","lineNumber":99,"sourceCode":"\n        out_weight *= self.dora_scale / direction_norm\n\n        return out_weight - orig_weight\n\n    def to(self, device: torch.device | None = None, dtype: torch.dtype | None = None):\n        super().to(device=device, dtype=dtype)\n        self.up = self.up.to(device=device, dtype=dtype)\n        self.down = self.down.to(device=device, dtype=dtype)\n        self.dora_scale = self.dora_scale.to(device=device, dtype=dtype)\n\n    def calc_size(self) -> int:\n        return super().calc_size() + calc_tensors_size([self.up, self.down, self.dora_scale])\n\n    def get_parameters(self, orig_parameters: dict[str, torch.Tensor], weight: float) -> dict[str, torch.Tensor]:\n        if any(p.device.type == \"meta\" for p in orig_parameters.values()):\n            # If any of the original parameters are on the 'meta' device, we assume this is because the base model is in\n            # a quantization format that doesn't allow easy dequantization.\n            raise RuntimeError(\n                \"The base model quantization format (likely bitsandbytes) is not compatible with DoRA patches.\"\n            )\n\n        scale = self.scale()\n        params = {\"weight\": self.get_weight(orig_parameters[\"weight\"]) * weight}\n        bias = self.get_bias(orig_parameters.get(\"bias\", None))\n        if bias is not None:\n            params[\"bias\"] = bias * (weight * scale)\n\n        # Reshape all params to match the original module's shape.\n        for param_name, param_weight in params.items():\n            orig_param = orig_parameters[param_name]\n            if param_weight.shape != orig_param.shape:\n                params[param_name] = param_weight.reshape(orig_param.shape)\n\n        return params\n","sourceCodeStart":81,"sourceCodeEnd":116,"githubUrl":"https://github.com/invoke-ai/InvokeAI/blob/0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06/invokeai/backend/patches/layers/dora_layer.py#L81-L116","documentation":"DoRALayer.get_parameters() needs the original layer weights to compute the DoRA weight decomposition. If any original parameter sits on the 'meta' device, the base weights were never materialized — typical of bitsandbytes quantization — so the required tensors are unavailable and RuntimeError is raised.","triggerScenarios":"Applying a DoRA LoRA to a model whose linear layers are quantized with bitsandbytes (or another format leaving weights on the meta device), causing `any(p.device.type == 'meta' ...)` to be True inside get_parameters().","commonSituations":"Loading a 4-bit/8-bit bnb-quantized checkpoint and then applying a DoRA adapter downloaded from the Hub; migrating LoRA configs between a standard model run and a quantized model run without noticing the adapter is DoRA.","solutions":["Load the base model unquantized (fp16) so DoRA weights can be computed.","Swap the DoRA adapter for a plain LoRA, which supports quantized bases.","Check the adapter's config for use_dora: true and convert it to a standard LoRA (dora removal tooling / retraining without DoRA).","Verify base-model quantization format in the Model Manager before attaching DoRA patches."],"exampleFix":"// before: bnb-quantized base + DoRA adapter\nmodel = load_model(path, quantization='bnb-nf4'); apply_dora(model, dora_lora)\n// after\nmodel = load_model(path, dtype=torch.float16); apply_dora(model, dora_lora)","handlingStrategy":"validation","validationCode":"meta_params = [n for n, p in orig_parameters.items() if p.device.type == 'meta']\nif meta_params:\n    print(f'Base weights not materialized ({meta_params}); DoRA unsupported — load model unquantized or use plain LoRA')","typeGuard":"def dora_compatible(orig_parameters: dict) -> bool:\n    return not any(p.device.type == 'meta' for p in orig_parameters.values())","tryCatchPattern":"try:\n    params = dora_layer.get_parameters(orig_parameters, weight)\nexcept RuntimeError as e:\n    if 'not compatible with DoRA' in str(e):\n        params = plain_lora_layer.get_parameters(orig_parameters, weight)  # fallback to LoRA\n    else:\n        raise","preventionTips":["Check adapter config for use_dora before attaching to a quantized base model.","Load bnb-quantized models in fp16 when applying DoRA adapters.","Prefer plain LoRA adapters for quantized inference workflows.","Inspect param devices (p.device.type != 'meta') after model load as a smoke test."],"tags":["dora","lora","quantization","bitsandbytes"],"backgroundTag":"quantization-incompatible-adapter","analyzedSha":"0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06","analyzedAt":"2026-08-29T04:46:49.967Z","schemaVersion":2},"datasetVersion":"2026-08-29T07:17:48.351Z"}