sgl-project/sglang · error · RuntimeError

Failed to cancel {len(errors)} VMM transport slice(s)

Error message

Failed to cancel {len(errors)} VMM transport slice(s)

What it means

cancel_for_dispatch cancels every wrapped VMM slice for outgoing items; if any individual pool.cancel_proxy call raises, the errors are collected and this aggregate RuntimeError is raised chained to the first failure. The commonest root cause is the per-slice 'no occupied slice at control offset' error (double release) or pool shutdown races.

Source

Thrown at python/sglang/srt/utils/cuda_vmm_transport_utils.py:1056

            return

        errors = []
        for item in mm_items:
            fields = (
                ("feature", item.feature),
                ("precomputed_embeddings", item.precomputed_embeddings),
            )
            for field, proxy in fields:
                if not isinstance(proxy, CudaVmmTensorTransportProxy):
                    continue
                try:
                    self.pool.cancel_proxy(proxy)
                except BaseException as error:
                    errors.append(error)
                finally:
                    setattr(item, field, None)
        if errors:
            raise RuntimeError(
                f"Failed to cancel {len(errors)} VMM transport slice(s)"
            ) from errors[0]

    def shutdown(self) -> None:
        if self.pool is None:
            return
        self.pool.shutdown()

View on GitHub (pinned to 0132848349)

Solutions

1. Inspect __cause__ of the raised error to identify the underlying per-slice failure and fix that (usually double-cancel)
2. Make cancellation idempotent: null the field before/after cancel (the finally already sets it to None) so re-dispatch cannot re-cancel
3. Add a released flag on proxies to short-circuit repeat cancels

try:
    transport.cancel_for_dispatch(items)
except RuntimeError as e:
    if "Failed to cancel" in str(e):
        log.warning("partial VMM cancel: %s", e.__cause__)
        # slices are nulled in finally; safe to proceed or rebuild

When it happens

Trigger: Any single slice cancellation failing during prepare_for_dispatch/_send_one_request/_send_batch_request — e.g. an item whose proxy was already cancelled or whose chunk was recycled; cancelling after pool shutdown began.

Common situations: Retry/error paths that re-dispatch items whose slices were already released; concurrent cancellation from multiple senders.

Related errors

• CUDA VMM POSIX FD broker failed
• CUDA VMM POSIX FD broker returned no file descriptor
• memory_size must be positive
• consumer_count must be positive
• recycle_interval must be positive

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f8e78bf785c671d9. Report an issue: GitHub.