sgl-project/sglang · error · RuntimeError

CUDA VMM pool has no occupied slice at control offset {contr

Error message

CUDA VMM pool has no occupied slice at control offset {control_offset}

What it means

Raised when cancel_proxy/cancel_from_pool is asked to release a reserved pool slice at a given control offset, but no occupied chunk in the CUDA VMM memory pool starts at that offset. This means the slice was already released, was never allocated, or the offset is stale/corrupted. It signals an accounting mismatch between the transport proxy bookkeeping and the pool's occupied chunk list.

Source

Thrown at python/sglang/srt/utils/cuda_vmm_transport_utils.py:535

    def _release_reserved_chunk(self, chunk: _CudaVmmMemoryChunk) -> None:
        if chunk in self.occupied_chunks:
            self.occupied_chunks.remove(chunk)
        self.available_chunks.append(_CudaVmmMemoryChunk(chunk.start, chunk.end))
        self._merge_chunks()

    def _cancel_control_offset(self, control_offset: int) -> None:
        with self._lock:
            chunk = next(
                (
                    chunk
                    for chunk in self.occupied_chunks
                    if chunk.start == control_offset
                ),
                None,
            )
            if chunk is None:
                raise RuntimeError(
                    "CUDA VMM pool has no occupied slice at control offset "
                    f"{control_offset}"
                )
            self._release_reserved_chunk(chunk)

    def cancel_proxy(self, proxy: CudaVmmTensorTransportProxy) -> None:
        """Return a published slice when its request was never dispatched."""
        if isinstance(proxy, CudaVmmPackedTensorTransportProxy):
            proxy._packed_owner.cancel_from_pool(self)
            return
        self._cancel_control_offset(proxy.control_offset)

    def _warn_pool_full_once(self, data_nbytes: int) -> None:
        if self._pool_full_warned:
            return
        self._pool_full_warned = True
        logger.warning(
            "CUDA VMM multimodal pool has no free chunk for a %.2f MiB tensor "

View on GitHub (pinned to 0132848349)

Solutions

Ensure cancel is called exactly once per slice (guard with a released flag or set the field to None immediately, as the finally block does)
Check chunk state / pool occupancy before cancelling: query whether an occupied chunk exists at the offset
Audit error paths in prepare_for_dispatch / _send_one_request for double-release after cancellation

Example fix

// before
pool.cancel_proxy(proxy)

// after
if proxy.released:
    return
pool.cancel_proxy(proxy)
proxy.released = True

Defensive patterns

Strategy: validation

Validate before calling

occupied = [c.start for c in pool.occupied_chunks]
if control_offset not in occupied:
    logger.warning("slice already released at %d", control_offset)
    return

Try / catch

try:
    pool.cancel_proxy(proxy)
except RuntimeError as e:
    if "no occupied slice" in str(e):
        return  # idempotent cancel
    raise

Prevention

Cancel each slice exactly once; null the field immediately after cancelling
Keep proxy release state on the proxy object and check it before cancel

When it happens

Trigger: Calling cancel_proxy() twice for the same proxy; using a CudaVmmTensorTransportProxy after its chunk was already released via acknowledge_consumption(); passing a control_offset that was never returned by the pool's reserve path.

Common situations: Double-cancel after a partial dispatch failure where some slices were already released; a request retry path that re-cancels items; racing shutdown() while cancels are in flight.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/edc4df790b100979. Report an issue: GitHub.