{"record":{"id":"4cdf205d5267c1b1","repo":"vllm-project/vllm","slug":"mm-tensor-ipc-torch-shm-is-not-supported-with-da","errorCode":null,"errorMessage":"mm_tensor_ipc='torch_shm' is not supported with data_parallel_size > 1 or tensor_parallel_size > 1 or pipeline_parallel_size > 1.","messagePattern":"mm_tensor_ipc='torch_shm' is not supported with data_parallel_size > 1 or tensor_parallel_size > 1 or pipeline_parallel_size > 1\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/model.py","lineNumber":1427,"sourceCode":"            if num_q_per_kv % decode_context_parallel_size != 0:\n                raise ValueError(\n                    \"The model's number of query heads per KV head \"\n                    f\"({num_q_per_kv}) must be divisible by \"\n                    \"`--decode-context-parallel-size` \"\n                    f\"({decode_context_parallel_size}) for GQA/MQA.\"\n                )\n\n        # torch_shm uses a single IPC queue to rank 0; DP>1 is\n        # incompatible because API servers can't know which\n        # CoreEngine the scheduler will assign work to. TP>1 is\n        # also not supported because this requires broadcasting\n        # MM tensors between all TP ranks.\n        if (\n            self.multimodal_config is not None\n            and self.multimodal_config.mm_tensor_ipc == \"torch_shm\"\n            and parallel_config.world_size_across_dp > 1\n        ):\n            raise ValueError(\n                \"mm_tensor_ipc='torch_shm' is not supported with \"\n                \"data_parallel_size > 1 or tensor_parallel_size > 1 \"\n                \"or pipeline_parallel_size > 1.\"\n            )\n\n    def get_sliding_window(self) -> int | None:\n        \"\"\"Get the sliding window size from the HF text config if present.\"\"\"\n        return getattr(self.hf_text_config, \"sliding_window\", None)\n\n    def get_vocab_size(self) -> int:\n        return self.model_arch_config.vocab_size\n\n    def get_hidden_size(self) -> int:\n        return self.model_arch_config.hidden_size\n\n    def get_inputs_embeds_size(self) -> int:\n        # The size of inputs_embeds is usually identical to the size\n        # of the hidden states, however there are exceptions, such as","sourceCodeStart":1409,"sourceCodeEnd":1445,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/model.py#L1409-L1445","documentation":"The torch_shm multimodal IPC path uses one shared-memory queue to rank 0, so it cannot work when the world size across data parallel exceeds 1 (DP>1 cannot route requests to the right engine; TP>1 would need broadcasting MM tensors). verify_with_parallel_config rejects the combination.","triggerScenarios":"multimodal_config.mm_tensor_ipc == 'torch_shm' together with parallel_config.world_size_across_dp > 1 (any of DP, TP, or PP greater than 1).","commonSituations":"Enabling the torch_shm fast path for multimodal input transfer on a single-GPU deployment, then scaling to multi-GPU TP without revisiting mm_tensor_ipc; setting it via env/config globally for a mixed fleet.","solutions":["Remove the torch_shm setting so the default multimodal IPC mechanism is used when running with DP/TP/PP > 1.","Keep mm_tensor_ipc='torch_shm' only on strictly single-GPU (world size 1) deployments.","If shared-memory transfer is required at scale, ask for/route via a supported IPC backend instead of torch_shm."],"exampleFix":"# before\nvllm serve llava-hf/llava-1.5-7b-hf --tensor-parallel-size 2 --mm-tensor-ipc torch_shm\n# after\nvllm serve llava-hf/llava-1.5-7b-hf --tensor-parallel-size 2","handlingStrategy":"validation","validationCode":"def mm_ipc_ok(mm_tensor_ipc: str | None, world_size_across_dp: int) -> bool:\n    return mm_tensor_ipc != 'torch_shm' or world_size_across_dp <= 1","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Treat mm_tensor_ipc='torch_shm' as a single-GPU-only tuning; strip it from any multi-GPU launch template.","Centralize IPC settings per deployment profile rather than per global config.","Assert world_size==1 in scripts that enable torch_shm."],"tags":["multimodal","ipc","parallelism","config"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}