huggingface/transformers · critical · ValueError

Invalid values: max_batch_tokens = {}, num_blocks = {}

Error message

Invalid values: max_batch_tokens = {}, num_blocks = {}

What it means

Raised by PagedAttentionCache._check_footprint when the solved values for max_batch_tokens or num_blocks are <= 0. The quadratic/linear solver produced a non-positive capacity, which means the memory budget cannot even accommodate the smallest useful batch — typically because fixed per-request overheads already consume all available_memory.

Source

Thrown at src/transformers/generation/continuous_batching/cache.py:738

            num_pages = floor((self.available_memory - cm * M - cmm * M**2) / (cn + cmn * M))
            num_blocks = num_pages // self.block_size

        elif max_batch_tokens is None:
            # N given → quadratic in M: coeff_mm·M² + (coeff_m + coeff_nm·N)·M + (coeff_n·N − avail) = 0
            N = num_blocks * self.block_size
            max_batch_tokens = int(self._solve_quadratic(cmm, cm + cmn * N, cn * N - self.available_memory))

        return max_batch_tokens, num_blocks

    def _check_footprint(self, max_batch_tokens: int, num_blocks: int) -> tuple[int, int]:
        """Checks if the footprint of the cache is within the available memory."""
        memory_footprint = self.compute_memory_footprint(max_batch_tokens, num_blocks)
        if memory_footprint > self.available_memory:
            raise MemoryError(
                f"Memory footprint {memory_footprint} is more than available memory {self.available_memory}"
            )
        if max_batch_tokens <= 0 or num_blocks <= 0:
            raise ValueError(f"Invalid values: max_batch_tokens = {max_batch_tokens}, num_blocks = {num_blocks}")
        return max_batch_tokens, num_blocks

    def _solve_quadratic(self, a: float, b: float, c: float) -> int:
        """Largest positive root of a·x² + b·x + c = 0. Falls back to linear when a == 0. Rounded down."""
        if a == 0:
            return int(-c / b)
        discriminant = b**2 - 4 * a * c
        if discriminant < 0:
            raise ValueError(f"No real solution (discriminant = {discriminant})")
        root = (-b + sqrt(discriminant)) / (2 * a)
        if root < 0:
            raise ValueError(f"No positive solution (root = {root})")
        return int(floor(root))

    # Formatting is disabled because of comment indentation, which improves readability.
    # fmt: off
    def _equation_coefficients(self, peak_deltas: tuple[int, ...]) -> tuple[int, ...]:
        """Given some deltas corresponding to an activation peak, returns the coefficients for the memory polynomial of

View on GitHub (pinned to a597f97485)

Solutions

1. Free GPU memory (delete prior tensors/managers, torch.cuda.empty_cache()) before creating the continuous-batching manager
2. Shrink the model or use quantization so weights leave room for the KV cache
3. Reduce cache dtype size (fp16/bf16 cache) or block_size so fixed overheads fit
4. If you passed explicit values, pass None to let auto-sizing pick feasible values

Example fix

# before
cfg = ContinuousBatchingConfig(max_batch_tokens=None)  # auto-size on a nearly-full GPU

# after
del old_manager
torch.cuda.empty_cache()
cfg = ContinuousBatchingConfig(max_batch_tokens=None)

free_b, _ = torch.cuda.mem_get_info()
if free_b < 1024**3:  # less than 1 GiB free cannot host any useful cache
    raise RuntimeError(f"Insufficient free memory: {free_b/1024**3:.2f} GiB")

try:
    manager = model.continuous_batching(config=cfg)
except ValueError as e:
    if 'Invalid values' in str(e):
        torch.cuda.empty_cache()  # reclaim and retry once
        manager = model.continuous_batching(config=cfg)
    else:
        raise

When it happens

Trigger: Cache sizing where available_memory minus constant terms (coeff_m·M, coeff_n·N at the chosen fill ratio) leaves nothing for tokens/blocks; extreme case: available_memory is 0 or negative because the model weights plus activations already exceed free memory.

Common situations: GPU nearly full before the manager is created; tiny available_memory with large block_size or many layers; unit tests with an artificially small available_memory value.

Related errors

• Memory footprint {} is more than available memory {}
• No real solution (discriminant = {})
• No positive solution (root = {})
• Failed to allocate {} blocks for request {}
• m must be provided if max_batch_tokens and num_blocks are No

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/5888c9957b4ef7c2. Report an issue: GitHub.