vllm-project/vllm · error · ValueError

mm_tensor_ipc='torch_shm' is not supported with data_paralle

Error message

mm_tensor_ipc='torch_shm' is not supported with data_parallel_size > 1 or tensor_parallel_size > 1 or pipeline_parallel_size > 1.

What it means

The torch_shm multimodal IPC path uses one shared-memory queue to rank 0, so it cannot work when the world size across data parallel exceeds 1 (DP>1 cannot route requests to the right engine; TP>1 would need broadcasting MM tensors). verify_with_parallel_config rejects the combination.

Source

Thrown at vllm/config/model.py:1427

            if num_q_per_kv % decode_context_parallel_size != 0:
                raise ValueError(
                    "The model's number of query heads per KV head "
                    f"({num_q_per_kv}) must be divisible by "
                    "`--decode-context-parallel-size` "
                    f"({decode_context_parallel_size}) for GQA/MQA."
                )

        # torch_shm uses a single IPC queue to rank 0; DP>1 is
        # incompatible because API servers can't know which
        # CoreEngine the scheduler will assign work to. TP>1 is
        # also not supported because this requires broadcasting
        # MM tensors between all TP ranks.
        if (
            self.multimodal_config is not None
            and self.multimodal_config.mm_tensor_ipc == "torch_shm"
            and parallel_config.world_size_across_dp > 1
        ):
            raise ValueError(
                "mm_tensor_ipc='torch_shm' is not supported with "
                "data_parallel_size > 1 or tensor_parallel_size > 1 "
                "or pipeline_parallel_size > 1."
            )

    def get_sliding_window(self) -> int | None:
        """Get the sliding window size from the HF text config if present."""
        return getattr(self.hf_text_config, "sliding_window", None)

    def get_vocab_size(self) -> int:
        return self.model_arch_config.vocab_size

    def get_hidden_size(self) -> int:
        return self.model_arch_config.hidden_size

    def get_inputs_embeds_size(self) -> int:
        # The size of inputs_embeds is usually identical to the size
        # of the hidden states, however there are exceptions, such as

View on GitHub (pinned to c794754062)

Solutions

1. Remove the torch_shm setting so the default multimodal IPC mechanism is used when running with DP/TP/PP > 1.
2. Keep mm_tensor_ipc='torch_shm' only on strictly single-GPU (world size 1) deployments.
3. If shared-memory transfer is required at scale, ask for/route via a supported IPC backend instead of torch_shm.

Example fix

# before
vllm serve llava-hf/llava-1.5-7b-hf --tensor-parallel-size 2 --mm-tensor-ipc torch_shm
# after
vllm serve llava-hf/llava-1.5-7b-hf --tensor-parallel-size 2

def mm_ipc_ok(mm_tensor_ipc: str | None, world_size_across_dp: int) -> bool:
    return mm_tensor_ipc != 'torch_shm' or world_size_across_dp <= 1

When it happens

Trigger: multimodal_config.mm_tensor_ipc == 'torch_shm' together with parallel_config.world_size_across_dp > 1 (any of DP, TP, or PP greater than 1).

Common situations: Enabling the torch_shm fast path for multimodal input transfer on a single-GPU deployment, then scaling to multi-GPU TP without revisiting mm_tensor_ipc; setting it via env/config globally for a mixed fleet.

Related errors

• Number of experts in the model must be greater than 0 when e
• Decode context parallelism for GQA/MQA requires `--tensor-pa
• `--decode-context-parallel-size` ({decode_context_parallel_s
• The model's number of query heads per KV head ({num_q_per_kv
• The model is not multimodal.

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/4cdf205d5267c1b1. Report an issue: GitHub.