sgl-project/sglang · error · RuntimeError

pool slot generation exhausted

Error message

{self.transport_name} pool slot generation exhausted

What it means

Each pool slot carries a 31-bit monotonically increasing generation counter used to detect stale leases (ABA protection). After 2^31-1 allocations of the same slot the counter would overflow, so the pool refuses further allocation.

Solutions

1. Restart the worker process when approaching the limit (operationally the cleanest fix)
2. Report upstream to sglang if genuinely hit — the counter width may need widening to 64-bit
3. Reduce allocation churn (larger batches, fewer copies) to slow generation growth

try:
    lease = pool.copy_tensor(t)
except RuntimeError as e:
    if 'generation exhausted' in str(e):
        signal_worker_restart()  # operational fallback
    else:
        raise

When it happens

Trigger: _allocate_locked (via copy_tensor) called ~2.1 billion times that reuse the same slot — practically only long-running servers with very high allocation churn or a pathological recycle loop.

Common situations: Ultra-long-lived serving process (weeks/months) recycling the same slot every iteration; stress tests that allocate/release in a tight loop.

Related errors

• byte_tensor must be a sufficiently large contiguous uint8…
• Cannot release inactive
• recycle_interval must be positive
• pool is too small after control metadata: pool= , control=
• A CUDA VMM-enabled model must provide a multimodal processor

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d317b25cfb0662df. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/multimodal/transport/memory_pool.py:257

            return len(self._occupied)

    def _allocate_locked(self, nbytes: int) -> Optional[PoolLease]:
        allocation_bytes = align_up(nbytes, DATA_ALIGNMENT)
        candidates = [
            (end - start, index, start, end)
            for index, (start, end) in enumerate(self._available_ranges)
            if end - start >= allocation_bytes
        ]
        if not candidates or not self._available_slots:
            return None
        _, index, start, end = min(candidates)
        self._available_ranges.pop(index)
        if start + allocation_bytes < end:
            self._available_ranges.append((start + allocation_bytes, end))
        slot = self._available_slots.pop()
        generation = self._slot_generations[slot] + 1
        if generation > 0x7FFFFFFF:
            raise RuntimeError(f"{self.transport_name} pool slot generation exhausted")
        self._slot_generations[slot] = generation
        ready_byte_offset = slot * self.control_words_per_slot * CONTROL_WORD_BYTES
        lease = PoolLease(
            start=start,
            end=start + allocation_bytes,
            nbytes=nbytes,
            slot=slot,
            generation=generation,
            ready_byte_offset=ready_byte_offset,
            ack_byte_offset=ready_byte_offset + CONTROL_WORD_BYTES,
        )
        self._occupied[slot] = lease
        return lease

    def _release_locked(self, lease: PoolLease) -> None:
        active_lease = self._occupied.get(lease.slot)
        if active_lease != lease:
            raise RuntimeError(

View on GitHub (pinned to 0132848349)