{"record":{"id":"5888c9957b4ef7c2","repo":"huggingface/transformers","slug":"invalid-values-max-batch-tokens-num-blocks","errorCode":null,"errorMessage":"Invalid values: max_batch_tokens = {}, num_blocks = {}","messagePattern":"Invalid values: max_batch_tokens = (.+?), num_blocks = (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"src/transformers/generation/continuous_batching/cache.py","lineNumber":738,"sourceCode":"            num_pages = floor((self.available_memory - cm * M - cmm * M**2) / (cn + cmn * M))\n            num_blocks = num_pages // self.block_size\n\n        elif max_batch_tokens is None:\n            # N given → quadratic in M: coeff_mm·M² + (coeff_m + coeff_nm·N)·M + (coeff_n·N − avail) = 0\n            N = num_blocks * self.block_size\n            max_batch_tokens = int(self._solve_quadratic(cmm, cm + cmn * N, cn * N - self.available_memory))\n\n        return max_batch_tokens, num_blocks\n\n    def _check_footprint(self, max_batch_tokens: int, num_blocks: int) -> tuple[int, int]:\n        \"\"\"Checks if the footprint of the cache is within the available memory.\"\"\"\n        memory_footprint = self.compute_memory_footprint(max_batch_tokens, num_blocks)\n        if memory_footprint > self.available_memory:\n            raise MemoryError(\n                f\"Memory footprint {memory_footprint} is more than available memory {self.available_memory}\"\n            )\n        if max_batch_tokens <= 0 or num_blocks <= 0:\n            raise ValueError(f\"Invalid values: max_batch_tokens = {max_batch_tokens}, num_blocks = {num_blocks}\")\n        return max_batch_tokens, num_blocks\n\n    def _solve_quadratic(self, a: float, b: float, c: float) -> int:\n        \"\"\"Largest positive root of a·x² + b·x + c = 0. Falls back to linear when a == 0. Rounded down.\"\"\"\n        if a == 0:\n            return int(-c / b)\n        discriminant = b**2 - 4 * a * c\n        if discriminant < 0:\n            raise ValueError(f\"No real solution (discriminant = {discriminant})\")\n        root = (-b + sqrt(discriminant)) / (2 * a)\n        if root < 0:\n            raise ValueError(f\"No positive solution (root = {root})\")\n        return int(floor(root))\n\n    # Formatting is disabled because of comment indentation, which improves readability.\n    # fmt: off\n    def _equation_coefficients(self, peak_deltas: tuple[int, ...]) -> tuple[int, ...]:\n        \"\"\"Given some deltas corresponding to an activation peak, returns the coefficients for the memory polynomial of","sourceCodeStart":720,"sourceCodeEnd":756,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/continuous_batching/cache.py#L720-L756","documentation":"Raised by PagedAttentionCache._check_footprint when the solved values for max_batch_tokens or num_blocks are <= 0. The quadratic/linear solver produced a non-positive capacity, which means the memory budget cannot even accommodate the smallest useful batch — typically because fixed per-request overheads already consume all available_memory.","triggerScenarios":"Cache sizing where available_memory minus constant terms (coeff_m·M, coeff_n·N at the chosen fill ratio) leaves nothing for tokens/blocks; extreme case: available_memory is 0 or negative because the model weights plus activations already exceed free memory.","commonSituations":"GPU nearly full before the manager is created; tiny available_memory with large block_size or many layers; unit tests with an artificially small available_memory value.","solutions":["Free GPU memory (delete prior tensors/managers, torch.cuda.empty_cache()) before creating the continuous-batching manager","Shrink the model or use quantization so weights leave room for the KV cache","Reduce cache dtype size (fp16/bf16 cache) or block_size so fixed overheads fit","If you passed explicit values, pass None to let auto-sizing pick feasible values"],"exampleFix":"# before\ncfg = ContinuousBatchingConfig(max_batch_tokens=None)  # auto-size on a nearly-full GPU\n\n# after\ndel old_manager\ntorch.cuda.empty_cache()\ncfg = ContinuousBatchingConfig(max_batch_tokens=None)","handlingStrategy":"validation","validationCode":"free_b, _ = torch.cuda.mem_get_info()\nif free_b < 1024**3:  # less than 1 GiB free cannot host any useful cache\n    raise RuntimeError(f\"Insufficient free memory: {free_b/1024**3:.2f} GiB\")","typeGuard":null,"tryCatchPattern":"try:\n    manager = model.continuous_batching(config=cfg)\nexcept ValueError as e:\n    if 'Invalid values' in str(e):\n        torch.cuda.empty_cache()  # reclaim and retry once\n        manager = model.continuous_batching(config=cfg)\n    else:\n        raise","preventionTips":["Check free memory before manager creation","Use a smaller/quantized model on small GPUs","Avoid passing explicit num_blocks on memory-constrained devices"],"tags":["memory","continuous-batching","configuration","solver"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}