{"record":{"id":"7790d620c26d7671","repo":"huggingface/transformers","slug":"got-max-requests-per-batch-but-expected-a-val","errorCode":null,"errorMessage":"Got {max_requests_per_batch = } but expected a value >= 1","messagePattern":"Got (.+?) but expected a value >= 1","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/continuous_batching/scheduler.py","lineNumber":42,"sourceCode":"    Abstract base class for scheduling requests in the continuous batch processor. Schedulers manage the lifecycle of\n    requests from when they are added to the waiting queue to when they are scheduled for processing. Different\n    schedulers implement different strategies for prioritizing and batching requests.\n    \"\"\"\n\n    def __init__(self, cache: PagedAttentionCache, safety_margin: float, max_requests_per_batch: int):\n        \"\"\"Initializes the scheduler. The safety margin is the percentage of free blocks under which we stop\n        scheduling new prefill requests, so safety_margin = 0.1 means that when there is less than 10% of free blocks,\n        or equivalently when more than 90% of blocks are already allocated, we stop scheduling new prefill requests.\n        Setting safety_margin to 0.0 means no safety margin is applied.\"\"\"\n        self.cache = cache\n        self.safety_margin = safety_margin\n        self.max_requests_per_batch = max_requests_per_batch\n        self._cancellation_lock = threading.Lock()\n        # Check args\n        if safety_margin < 0 or safety_margin > 1:\n            raise ValueError(f\"Got {safety_margin = } but expected a value in [0, 1]\")\n        if max_requests_per_batch < 1:\n            raise ValueError(f\"Got {max_requests_per_batch = } but expected a value >= 1\")\n        # This is to compute the read cache used by a new request being scheduled\n        self.read_cache_limit = None if self.cache.num_full_attention_groups else self.cache.config.sliding_window\n        self.max_decode_fast_path_length = self.cache.max_blocks_per_request * self.cache.block_size\n        # Initialize mutable states via reset()\n        self.reset()\n\n    def reset(self) -> None:\n        \"\"\"Reset scheduler state for a new generation loop.\"\"\"\n        self.active_requests: dict[str, RequestState] = {}\n        self.waiting_requests: dict[str, RequestState] = {}\n        self.waiting_requests_order: deque[str] = deque()\n        self._requests_to_cancel: set[str] = set()\n        self._requests_to_fork: list[RequestState] = []\n        self.block_new_requests = False\n        # Active requests that failed block allocation in the last scheduled batch, with their physical block demand.\n        # The offloading manager uses this to size bulk evictions.\n        self.starved_requests: list[tuple[RequestState, int]] = []\n","sourceCodeStart":24,"sourceCodeEnd":60,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/continuous_batching/scheduler.py#L24-L60","documentation":"Scheduler.__init__ validates max_requests_per_batch >= 1: zero or negative batch sizes are meaningless and rejected. This bounds how many requests share one scheduler step.","triggerScenarios":"Passing max_requests_per_batch=0 in ContinuousBatchingConfig — commonly from a computed value (num_gpus - 1 with 1 GPU, a config default of 0, or math gone wrong).","commonSituations":"Dynamic configs computing the value from environment variables (empty string parsed as 0); scaling experiments sweeping to 0; copy of a template with a placeholder 0.","solutions":["Set max_requests_per_batch to at least 1 (typically 16-256 depending on hardware)","If the value is computed, clamp it: max(1, computed_value)","Audit env-var parsing that can yield 0 for unset variables"],"exampleFix":"# before\nn = int(os.environ.get('MAX_REQ', 0))\ncfg = ContinuousBatchingConfig(max_requests_per_batch=n)\n\n# after\nn = max(1, int(os.environ.get('MAX_REQ', 32)))\ncfg = ContinuousBatchingConfig(max_requests_per_batch=n)","handlingStrategy":"validation","validationCode":"cfg.max_requests_per_batch = max(1, int(cfg.max_requests_per_batch))","typeGuard":"def is_valid_max_requests(v) -> bool:\n    return isinstance(v, int) and v >= 1","tryCatchPattern":null,"preventionTips":["Clamp computed values with max(1, x)","Give env-var lookups sane non-zero defaults","Validate config objects after deserialization"],"tags":["validation","scheduler","configuration","continuous-batching"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}