sgl-project/sglang · error · ValueError

{transport_name} pool is too small after control metadata: p

Error message

{transport_name} pool is too small after control metadata: pool={memory_size}, control={self.data_start}

What it means

The pool reserves its leading bytes for per-slice control metadata (max_inflight_slices slots of control words, aligned up to DATA_ALIGNMENT) before payload data. If memory_size is not larger than this control region, no usable payload space remains and construction aborts.

Source

Thrown at python/sglang/srt/multimodal/transport/memory_pool.py:204

            raise ValueError(
                "byte_tensor must be a sufficiently large contiguous uint8 tensor "
                f"on cuda:{device_id}"
            )

        self.memory_size = memory_size
        self.byte_tensor = byte_tensor
        self.base_address = base_address
        self.device_id = device_id
        self.consumer_count = consumer_count
        self.control_words_per_slot = 1 + consumer_count
        self.max_inflight_slices = max_inflight_slices
        self.transport_name = transport_name
        control_bytes = (
            max_inflight_slices * self.control_words_per_slot * CONTROL_WORD_BYTES
        )
        self.data_start = align_up(control_bytes, DATA_ALIGNMENT)
        if memory_size <= self.data_start:
            raise ValueError(
                f"{transport_name} pool is too small after control metadata: "
                f"pool={memory_size}, control={self.data_start}"
            )

        control_word_count = max_inflight_slices * self.control_words_per_slot
        self._control_words = (
            byte_tensor[: control_word_count * CONTROL_WORD_BYTES]
            .view(torch.int32)
            .view(max_inflight_slices, self.control_words_per_slot)
        )
        self._control_words.zero_()
        torch.cuda.synchronize(device_id)

        self._available_ranges = [(self.data_start, memory_size)]
        self._available_slots = list(reversed(range(max_inflight_slices)))
        self._slot_generations = [0] * max_inflight_slices
        self._occupied: dict[int, PoolLease] = {}
        self._lock = threading.Lock()

View on GitHub (pinned to 0132848349)

Solutions

1. Increase memory_size well beyond data_start (e.g. 2-4x the largest expected inflight payload total)
2. Reduce max_inflight_slices if high slice concurrency is not needed
3. Compute control_bytes = max_inflight_slices * control_words_per_slot * CONTROL_WORD_BYTES plus alignment and assert headroom before constructing

Example fix

# before
pool = SharedMemoryPool(memory_size=1<<20, max_inflight_slices=64, ...)
# after
pool = SharedMemoryPool(memory_size=64<<20, max_inflight_slices=64, ...)

control = align_up(max_inflight_slices * control_words_per_slot * CONTROL_WORD_BYTES, DATA_ALIGNMENT)
assert memory_size > control * 4, 'pool too small for control metadata + payload'

When it happens

Trigger: Small memory_size combined with large max_inflight_slices; e.g. memory_size=1MB with max_inflight_slices=64 can be fully consumed by control words plus alignment padding.

Common situations: Sizing the pool with a rough 'a few MB' guess; raising max_inflight_slices without growing the pool; shrinking memory_size for tests.

Related errors

• recycle_interval must be positive
• byte_tensor must be a sufficiently large contiguous uint8 te
• {self.transport_name} pool slot generation exhausted
• Cannot release inactive {self.transport_name} pool lease (sl
• cos/sin shape does not cover image tokens and head_dim

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/fcda2ee5a638df8c. Report an issue: GitHub.