sgl-project/sglang · error · RuntimeError

{self.transport_name} pool slot generation exhausted

Error message

{self.transport_name} pool slot generation exhausted

What it means

Each pool slot carries a 31-bit monotonically increasing generation counter used to detect stale leases (ABA protection). After 2^31-1 allocations of the same slot the counter would overflow, so the pool refuses further allocation.

Source

Thrown at python/sglang/srt/multimodal/transport/memory_pool.py:257

            return len(self._occupied)

    def _allocate_locked(self, nbytes: int) -> Optional[PoolLease]:
        allocation_bytes = align_up(nbytes, DATA_ALIGNMENT)
        candidates = [
            (end - start, index, start, end)
            for index, (start, end) in enumerate(self._available_ranges)
            if end - start >= allocation_bytes
        ]
        if not candidates or not self._available_slots:
            return None
        _, index, start, end = min(candidates)
        self._available_ranges.pop(index)
        if start + allocation_bytes < end:
            self._available_ranges.append((start + allocation_bytes, end))
        slot = self._available_slots.pop()
        generation = self._slot_generations[slot] + 1
        if generation > 0x7FFFFFFF:
            raise RuntimeError(f"{self.transport_name} pool slot generation exhausted")
        self._slot_generations[slot] = generation
        ready_byte_offset = slot * self.control_words_per_slot * CONTROL_WORD_BYTES
        lease = PoolLease(
            start=start,
            end=start + allocation_bytes,
            nbytes=nbytes,
            slot=slot,
            generation=generation,
            ready_byte_offset=ready_byte_offset,
            ack_byte_offset=ready_byte_offset + CONTROL_WORD_BYTES,
        )
        self._occupied[slot] = lease
        return lease

    def _release_locked(self, lease: PoolLease) -> None:
        active_lease = self._occupied.get(lease.slot)
        if active_lease != lease:
            raise RuntimeError(

View on GitHub (pinned to 0132848349)

Solutions

1. Restart the worker process when approaching the limit (operationally the cleanest fix)
2. Report upstream to sglang if genuinely hit — the counter width may need widening to 64-bit
3. Reduce allocation churn (larger batches, fewer copies) to slow generation growth

try:
    lease = pool.copy_tensor(t)
except RuntimeError as e:
    if 'generation exhausted' in str(e):
        signal_worker_restart()  # operational fallback
    else:
        raise

When it happens

Trigger: _allocate_locked (via copy_tensor) called ~2.1 billion times that reuse the same slot — practically only long-running servers with very high allocation churn or a pathological recycle loop.

Common situations: Ultra-long-lived serving process (weeks/months) recycling the same slot every iteration; stress tests that allocate/release in a tight loop.

Related errors

• recycle_interval must be positive
• byte_tensor must be a sufficiently large contiguous uint8 te
• {transport_name} pool is too small after control metadata: p
• Cannot release inactive {self.transport_name} pool lease (sl
• cos/sin shape does not cover image tokens and head_dim

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d317b25cfb0662df. Report an issue: GitHub.