{"record":{"id":"c45d039ea1e96d75","repo":"huggingface/transformers","slug":"got-safety-margin-but-expected-a-value-in-0","errorCode":null,"errorMessage":"Got {safety_margin = } but expected a value in [0, 1]","messagePattern":"Got (.+?) but expected a value in \\[0, 1\\]","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/continuous_batching/scheduler.py","lineNumber":40,"sourceCode":"class Scheduler(ABC):\n    \"\"\"\n    Abstract base class for scheduling requests in the continuous batch processor. Schedulers manage the lifecycle of\n    requests from when they are added to the waiting queue to when they are scheduled for processing. Different\n    schedulers implement different strategies for prioritizing and batching requests.\n    \"\"\"\n\n    def __init__(self, cache: PagedAttentionCache, safety_margin: float, max_requests_per_batch: int):\n        \"\"\"Initializes the scheduler. The safety margin is the percentage of free blocks under which we stop\n        scheduling new prefill requests, so safety_margin = 0.1 means that when there is less than 10% of free blocks,\n        or equivalently when more than 90% of blocks are already allocated, we stop scheduling new prefill requests.\n        Setting safety_margin to 0.0 means no safety margin is applied.\"\"\"\n        self.cache = cache\n        self.safety_margin = safety_margin\n        self.max_requests_per_batch = max_requests_per_batch\n        self._cancellation_lock = threading.Lock()\n        # Check args\n        if safety_margin < 0 or safety_margin > 1:\n            raise ValueError(f\"Got {safety_margin = } but expected a value in [0, 1]\")\n        if max_requests_per_batch < 1:\n            raise ValueError(f\"Got {max_requests_per_batch = } but expected a value >= 1\")\n        # This is to compute the read cache used by a new request being scheduled\n        self.read_cache_limit = None if self.cache.num_full_attention_groups else self.cache.config.sliding_window\n        self.max_decode_fast_path_length = self.cache.max_blocks_per_request * self.cache.block_size\n        # Initialize mutable states via reset()\n        self.reset()\n\n    def reset(self) -> None:\n        \"\"\"Reset scheduler state for a new generation loop.\"\"\"\n        self.active_requests: dict[str, RequestState] = {}\n        self.waiting_requests: dict[str, RequestState] = {}\n        self.waiting_requests_order: deque[str] = deque()\n        self._requests_to_cancel: set[str] = set()\n        self._requests_to_fork: list[RequestState] = []\n        self.block_new_requests = False\n        # Active requests that failed block allocation in the last scheduled batch, with their physical block demand.\n        # The offloading manager uses this to size bulk evictions.","sourceCodeStart":22,"sourceCodeEnd":58,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/continuous_batching/scheduler.py#L22-L58","documentation":"Scheduler.__init__ validates safety_margin: the fraction of free cache blocks below which new prefill requests stop being scheduled. Values outside [0, 1] are rejected — it is a percentage, not a token count or a byte count.","triggerScenarios":"Constructing the scheduler (via ContinuousBatchingConfig) with safety_margin=10 (user means '10%' but writes 10) or a negative value. 0.1 means stop scheduling when <10% blocks are free.","commonSituations":"Config files ported from other engines where the margin is expressed in blocks or permille; YAML parsing producing strings/ints >1; mental model mismatch with 'reserved blocks' semantics.","solutions":["Express the margin as a fraction in [0, 1]: 0.1 for 10%","Use 0.0 only if you deliberately want no safety margin","Validate config values after loading from YAML/JSON before passing them in"],"exampleFix":"# before\ncfg = ContinuousBatchingConfig(safety_margin=10)\n\n# after\ncfg = ContinuousBatchingConfig(safety_margin=0.1)","handlingStrategy":"validation","validationCode":"assert isinstance(cfg.safety_margin, (int, float)) and 0 <= cfg.safety_margin <= 1, \\\n    f'safety_margin must be a fraction in [0,1], got {cfg.safety_margin}'","typeGuard":"def is_valid_safety_margin(v) -> bool:\n    return isinstance(v, (int, float)) and 0 <= v <= 1","tryCatchPattern":null,"preventionTips":["Treat safety_margin as a percentage fraction, not a count","Validate loaded YAML/JSON config values before use"],"tags":["validation","scheduler","configuration","continuous-batching"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}