{"record":{"id":"e683c906f8196412","repo":"huggingface/transformers","slug":"cpu-offload-space-none-requires-psutil-to-auto-siz","errorCode":null,"errorMessage":"cpu_offload_space=None requires psutil to auto-size the CPU swap pool. Install psutil or pass an explicit GiB value.","messagePattern":"cpu_offload_space=None requires psutil to auto-size the CPU swap pool\\. Install psutil or pass an explicit GiB value\\.","errorType":"exception","errorClass":"ImportError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/continuous_batching/offloading_manager.py","lineNumber":156,"sourceCode":"                clamped_gib = max_bytes / (1024**3)\n                logger.warning(\n                    f\"cpu_offload_space={cpu_offload_space_gib:.1f} GiB exceeds {safety_threshold:.0%} of total RAM \"\n                    f\"({total_ram / (1024**3):.1f} GiB). Clamping to {clamped_gib:.1f} GiB.\"\n                )\n                offload_bytes = max_bytes\n        # Else if the max is None, throw a warning and accept the requested number of bytes as is\n        elif offload_bytes is not None:\n            logger.warning(\n                \"psutil is not available — cpu_offload_space_safety_threshold cannot be enforced. \"\n                \"Install psutil to enable the safety cap.\"\n            )\n        # Else if the requested number of bytes is None, we use the max number of bytes as the requested number of bytes\n        elif max_bytes is not None:\n            offload_bytes = max_bytes\n            logger.warning(f\"Auto-sizing CPU swap pool from safety threshold: {max_bytes / (1024**3):.2f} GiB.\")\n        # Otherwise, it means the pool was supposed to be sized using psutil but it is not available\n        else:\n            raise ImportError(\n                \"cpu_offload_space=None requires psutil to auto-size the CPU swap pool. Install psutil or pass an \"\n                \"explicit GiB value.\"\n            )\n\n        # Compute how many blocks fit in CPU pool\n        bytes_per_block = (\n            2                                 # one for key, one for value\n            * len(self.cache.key_cache)       # number of layers in a layer group\n            * self.cache.block_size           # block size\n            * self.cache.num_key_value_heads  # number of key value heads\n            * self.cache.head_dim             # head dimension\n            * self.cache.dtype.itemsize       # data type size in bytes\n        )  # fmt: skip\n        if bytes_per_block == 0:\n            raise ValueError(\"The number of bytes per block is 0. This is not possible.\")\n        return offload_bytes // bytes_per_block\n\n    def _stream_ctx(self):","sourceCodeStart":138,"sourceCodeEnd":174,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/continuous_batching/offloading_manager.py#L138-L174","documentation":"Raised in OffloadingManager._compute_num_cpu_blocks when cpu_offload_space is None (auto-size requested) but psutil is not installed. Auto-sizing reads available RAM via psutil.virtual_memory(); without it there is no safe way to pick a pool size, so it fails rather than guessing.","triggerScenarios":"Constructing a ContinuousBatchingConfig/manager with cpu_offload_space=None (the auto default) on an environment where import psutil fails — slim Docker images, minimal CI images, or transformers installed without the psutil extra.","commonSituations":"Docker images without psutil; new venv where transformers[torch] was installed but not psutil; CI pipeline using the same config as a production box that has psutil.","solutions":["pip install psutil (or install transformers with the extra that includes it)","Pass an explicit value: ContinuousBatchingConfig(cpu_offload_space=8.0) (GiB)","For Dockerfiles, add psutil next to transformers"],"exampleFix":"# before (no psutil installed)\ncfg = ContinuousBatchingConfig(cpu_offload_space=None)\n\n# after\n# option A: pip install psutil\n# option B:\ncfg = ContinuousBatchingConfig(cpu_offload_space=8.0)","handlingStrategy":"validation","validationCode":"try:\n    import psutil  # noqa\n    cpu_offload_space = None          # auto-size is safe\nexcept ImportError:\n    cpu_offload_space = 8.0           # explicit GiB fallback\ncfg = ContinuousBatchingConfig(cpu_offload_space=cpu_offload_space)","typeGuard":null,"tryCatchPattern":"try:\n    manager = model.continuous_batching(config=cfg)\nexcept ImportError as e:\n    if 'psutil' in str(e):\n        cfg.cpu_offload_space = 8.0\n        manager = model.continuous_batching(config=cfg)\n    else:\n        raise","preventionTips":["Add psutil to inference Dockerfiles/requirements","Pin an explicit cpu_offload_space in environments you don't control"],"tags":["dependencies","psutil","offloading","continuous-batching"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}