{"record":{"id":"3ecacf75766e2c17","repo":"huggingface/transformers","slug":"m-must-be-provided-if-max-batch-tokens-and-num-blo","errorCode":null,"errorMessage":"m must be provided if max_batch_tokens and num_blocks are None","messagePattern":"m must be provided if max_batch_tokens and num_blocks are None","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/continuous_batching/cache.py","lineNumber":710,"sourceCode":"        final_n = min([solution[1] for solution in solutions])\n        return final_m, final_n\n\n    def _solve_for_peak(\n        self,\n        peak: tuple[int, ...],\n        max_batch_tokens: int | None,\n        num_blocks: int | None,\n        cache_fill_per_batch: float | None,\n    ) -> tuple[int, int]:\n        \"\"\"Returns a couple of `(max_batch_tokens, num_blocks)` that satisfy the memory constraint for the given\n        activation peak.\"\"\"\n        cm, cn, cmn, cmm = self._equation_coefficients(peak)\n\n        # If neither variable is defined, use a quadratic solver\n        if max_batch_tokens is None and num_blocks is None:\n            # Substitute M = m·N → (coeff_nm·m + coeff_mm·m²)·N² + (coeff_n + coeff_m·m)·N − avail = 0\n            if cache_fill_per_batch is None:\n                raise ValueError(\"m must be provided if max_batch_tokens and num_blocks are None\")\n            m = cache_fill_per_batch  # as in, m is a substitute for big M, which is max_batch_tokens\n            num_pages = self._solve_quadratic(cmn * m + cmm * m**2, cn + cm * m, -self.available_memory)\n            max_batch_tokens = int(num_pages * m)\n            num_blocks = int(num_pages) // self.block_size\n\n        # Otherwise, use a linear solver\n        elif num_blocks is None:\n            # M given → linear in N: (coeff_n + coeff_nm·M)·N = avail − coeff_m·M − coeff_mm·M²\n            M = max_batch_tokens\n            num_pages = floor((self.available_memory - cm * M - cmm * M**2) / (cn + cmn * M))\n            num_blocks = num_pages // self.block_size\n\n        elif max_batch_tokens is None:\n            # N given → quadratic in M: coeff_mm·M² + (coeff_m + coeff_nm·N)·M + (coeff_n·N − avail) = 0\n            N = num_blocks * self.block_size\n            max_batch_tokens = int(self._solve_quadratic(cmm, cm + cmn * N, cn * N - self.available_memory))\n\n        return max_batch_tokens, num_blocks","sourceCodeStart":692,"sourceCodeEnd":728,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/continuous_batching/cache.py#L692-L728","documentation":"ValueError from PagedAttentionMemoryHandler's solver: when both max_batch_tokens and num_blocks are None, the memory equation is closed by substituting M = m*N (max_batch_tokens = cache_fill_per_batch * num_blocks); without a ratio m the quadratic has two unknowns and cannot be solved. cache_fill_per_batch encodes the target ratio between activation memory and KV cache memory.","triggerScenarios":"Calling the memory planner / resolve_max_memory_percent with both max_batch_tokens=None and num_blocks=None and no cache_fill_per_batch; a ContinuousBatchingConfig where none of the three fields is set and automatic sizing kicks in.","commonSituations":"Relying on automatic memory sizing on a new setup where the default fill ratio was not propagated; constructing the handler directly in tests or custom servers.","solutions":["Set cache_fill_per_batch (the m ratio) so the solver can size the cache, e.g. ContinuousBatchingConfig(cache_fill_per_batch=8)","Or provide max_batch_tokens and let the solver derive num_blocks linearly","Or provide num_blocks directly and let max_batch_tokens be derived"],"exampleFix":"# before\ncb_cfg = ContinuousBatchingConfig(max_batch_tokens=None, num_blocks=None, cache_fill_per_batch=None)\n# after\ncb_cfg = ContinuousBatchingConfig(cache_fill_per_batch=8)","handlingStrategy":"validation","validationCode":"if cb_cfg.max_batch_tokens is None and cb_cfg.num_blocks is None:\n    assert cb_cfg.cache_fill_per_batch is not None, 'set cache_fill_per_batch when auto-sizing memory'\n    cb_cfg.cache_fill_per_batch = cb_cfg.cache_fill_per_batch or 8","typeGuard":"def memory_plan_is_solvable(cb_cfg) -> bool:\n    return cb_cfg.max_batch_tokens is not None or cb_cfg.num_blocks is not None or cb_cfg.cache_fill_per_batch is not None","tryCatchPattern":null,"preventionTips":["Always set at least one of max_batch_tokens, num_blocks, or cache_fill_per_batch","Use cache_fill_per_batch to express the activation-vs-KV memory ratio for auto-sizing"],"tags":["continuous-batching","memory-planning","configuration","validation"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}