{"record":{"id":"ee3500151495ce19","repo":"labmlai/annotated_deep_learning_paper_implementations","slug":"please-install-bitsandbytes-with-pip-install-bi","errorCode":null,"errorMessage":"Please install `bitsandbytes` with `pip install bitsandbytes -U`","messagePattern":"Please install `bitsandbytes` with `pip install bitsandbytes -U`","errorType":"exception","errorClass":"ImportError","httpStatus":null,"severity":"error","filePath":"labml_nn/neox/utils/llm_int8.py","lineNumber":37,"sourceCode":"These features get clamped in 8-bit integer space which causes the model performance to degrade.\nAs a solution they pick these outliers (greater than a specified threshold)\nand compute their multiplications separately in float16 space.\nSince the percentage of outliers is around 0.01% this doesn't increase memory usage,\nand prevents the model from degrading performance.\n\nThe code to transform GPT-NoeX layers is defined in [model.py](../model.html#post_load_prepare).\n\nHere are example uses of GPT-NeoX with int8 quantization.\n\n* [Generate Text](../samples/llm_int8.html)\n* [Run Evaluation Tests](../evaluation/llm_int8.html)\n\"\"\"\n\n# Import [`bitsandbytes`](https://github.com/timdettmers/bitsandbytes) package\ntry:\n    from bitsandbytes.nn import Linear8bitLt, Int8Params\nexcept ImportError:\n    raise ImportError('''Please install `bitsandbytes` with `pip install bitsandbytes -U`''')\n\nimport torch\nfrom torch import nn\n\n\ndef make_llm_int8_linear(linear_module: nn.Linear, device: torch.device, threshold: float = 6.0):\n    \"\"\"\n    ## Transform a `nn.Linear` layer to LLM.int8() linear layer\n\n    :param linear_module: is the `nn.Linear` layer to transform\n    :param device: is the device of the model\n    :param threshold: is the threshold $\\alpha$ to use for outlier detection\n    \"\"\"\n\n    #\n    assert isinstance(linear_module, nn.Linear)\n\n    # Create an empty Linear8bitLt module","sourceCodeStart":19,"sourceCodeEnd":55,"githubUrl":"https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/33ab02281c2b928e6b32792909cc79cbdcfe1d6a/labml_nn/neox/utils/llm_int8.py#L19-L55","documentation":"labml-nn's LLM.int8() support delegates 8-bit linear layers to the third-party bitsandbytes package. The module does a bare 'from bitsandbytes.nn import Linear8bitLt, Int8Params' at import time and converts any ImportError into an actionable ImportError telling you to install the package. It is purely an environment problem, not a code or data problem.","triggerScenarios":"Importing labml_nn.neox.utils.llm_int8 (or a module that imports it) in an environment where bitsandbytes is not installed or is broken enough that bitsandbytes.nn fails to import.","commonSituations":"Fresh conda/venv without extras installed; CI image that only installs labml-nn core deps; a bitsandbytes binary built for a different CUDA/torch version raising ImportError on import; dependency resolver having uninstalled it during an upgrade.","solutions":["pip install bitsandbytes -U (ideally pip install bitsandbytes -U --no-cache-dir)","Verify the install matches your torch/CUDA build; if import still fails, reinstall torch and bitsandbytes into the same environment","If you do not need int8 quantization, stop importing/making llm_int8 layers so the module is never loaded","Pre-flight check in your entrypoint: importlib.util.find_spec('bitsandbytes') before importing llm_int8"],"exampleFix":"# before: ImportError at import time\nfrom labml_nn.neox.utils.llm_int8 import make_llm_int8_linear\n\n# after: guard the optional dependency\nimport importlib.util\nif importlib.util.find_spec('bitsandbytes') is None:\n    raise SystemExit('bitsandbytes required: pip install bitsandbytes -U')\nfrom labml_nn.neox.utils.llm_int8 import make_llm_int8_linear","handlingStrategy":"validation","validationCode":"import importlib.util\nif importlib.util.find_spec('bitsandbytes') is None:\n    raise SystemExit('This script requires bitsandbytes: pip install bitsandbytes -U')\n\n# only now import the int8 module\nfrom labml_nn.neox.utils.llm_int8 import make_llm_int8_linear","typeGuard":"def has_bitsandbytes() -> bool:\n    import importlib.util\n    return importlib.util.find_spec('bitsandbytes') is not None","tryCatchPattern":"try:\n    from labml_nn.neox.utils.llm_int8 import make_llm_int8_linear\nexcept ImportError as e:\n    raise SystemExit(f'bitsandbytes missing or broken: {e}. Install with: pip install bitsandbytes -U') from e","preventionTips":["Pin extras in requirements: labml-nn[...] plus bitsandbytes for int8 runs","Smoke-test 'import bitsandbytes' in CI before the training job","Keep quantization dependencies in the same env as the matching torch/CUDA version"],"tags":["python","pytorch","bitsandbytes","quantization","dependency"],"backgroundTag":"missing-dependency","analyzedSha":"33ab02281c2b928e6b32792909cc79cbdcfe1d6a","analyzedAt":"2026-08-25T10:30:27.743Z","schemaVersion":2},"datasetVersion":"2026-08-25T11:17:15.655Z"}