{"record":{"id":"67143a7de17a32ea","repo":"vllm-project/vllm","slug":"only-dim-0-all-gatherv-is-supported","errorCode":null,"errorMessage":"only dim 0 all-gatherv is supported","messagePattern":"only dim 0 all-gatherv is supported","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"vllm/distributed/device_communicators/xpu_communicator.py","lineNumber":118,"sourceCode":"            output_shape, dtype=input_tensor.dtype, device=input_tensor.device\n        )\n        if sizes is not None and sizes.count(sizes[0]) != len(sizes):\n            # if inputs shape in different ranks is not the same using reduce_scatter\n            input_splits = list(input_tensor.split(sizes, dim=0))\n            dist.reduce_scatter(output, input_splits, group=self.device_group)\n        else:\n            dist.reduce_scatter_tensor(output, input_tensor, group=self.device_group)\n        # Reshape before returning\n        return output.movedim(0, dim).contiguous()\n\n    def all_gatherv(\n        self,\n        input_: torch.Tensor | list[torch.Tensor],\n        dim: int = 0,\n        sizes: list[int] | None = None,\n    ):\n        if dim != 0:\n            raise NotImplementedError(\"only dim 0 all-gatherv is supported\")\n        world_size = self.world_size\n\n        # 'sizes' is not needed if all inputs in the same group have the same\n        # shape\n        if sizes is not None and all(s == sizes[0] for s in sizes):\n            sizes = None\n\n        def _all_gather_single(input_: torch.Tensor, sizes: list[int] | None = None):\n            input_size = input_.size()\n            if sizes is not None:\n                assert len(sizes) == world_size\n                assert input_.shape[dim] == sizes[self.rank_in_group], (\n                    f\"{input_.shape[dim]} != {sizes[self.rank_in_group]}\"\n                )\n                output_size = (sum(sizes),) + input_size[1:]\n            else:\n                output_size = (input_size[0] * world_size,) + input_size[1:]\n            # Allocate output tensor.","sourceCodeStart":100,"sourceCodeEnd":136,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/distributed/device_communicators/xpu_communicator.py#L100-L136","documentation":"XPUCommunicator.all_gatherv() implements the variable-size all-gather only along dim 0. Gathering along any other dim would require either transposes or a dim-aware gather kernel that the oneDPL/SYCL backend path here does not provide, so dim != 0 raises NotImplementedError rather than silently producing wrong layout.","triggerScenarios":"Calling all_gatherv(input, dim=1) or dim=-1 on an Intel GPU (XPU) device group; running model or parallelism code that was written against the PyTorch/NVSHMEM communicator which supports arbitrary dims.","commonSituations":"Porting a model from CUDA to Intel GPU (vLLM XPU backend) where a tensor-parallel op gathers along a non-zero dim; default code paths that pass dim explicitly instead of moving the dim afterwards.","solutions":["Move the gather dimension to 0 first: call all_gatherv(input_.movedim(dim, 0).contiguous(), dim=0, sizes=sizes) and movedim back afterwards","Restructure the caller to keep the gathered axis first (dim 0 layout) so the dim=0 implementation applies","If the sizes are uniform, use plain all_gather_in_place/all_gather which may support the shape you need"],"exampleFix":"# before\nout = xpu_comm.all_gatherv(x, dim=1, sizes=sizes)\n# NotImplementedError: only dim 0 all-gatherv is supported\n\n# after\nout = xpu_comm.all_gatherv(x.movedim(1, 0).contiguous(), dim=0, sizes=sizes).movedim(0, 1)","handlingStrategy":"validation","validationCode":"dim = 0 if dim == 0 else None\nassert dim == 0, \"XPUCommunicator.all_gatherv supports dim=0 only; move the axis first\"","typeGuard":"def xpu_all_gatherv_ok(dim: int) -> bool:\n    return dim == 0","tryCatchPattern":null,"preventionTips":["Normalize tensors to dim-0 layout before variable-size gathers on XPU","Write dimension-agnostic wrappers that movedim() transparently","Add XPU-specific tests for any gather code ported from CUDA"],"tags":["xpu","distributed","unsupported-operation"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}