sgl-project/sglang · error · RuntimeError

auxiliary output does not support pipeline-parallel transpor

Error message

auxiliary output does not support pipeline-parallel transport

What it means

During pipeline-parallel sampling, a non-PP auxiliary output object was passed to the PP tensor transport helper; only PipelineParallelAuxiliaryOutput instances implement to_pp_tensors.

Source

Thrown at python/sglang/srt/sampling/sampling_observer_pp.py:42

@runtime_checkable
class PipelineParallelSamplingObserver(Protocol):
    def from_pp_tensors(
        self, tensors: Mapping[str, torch.Tensor]
    ) -> DeviceAuxiliaryOutput: ...


_OUTPUT_PREFIX = "__sampling_observer_output__."


def add_auxiliary_output_to_pp_tensors(
    tensors: MutableMapping[str, Any],
    output: Optional[DeviceAuxiliaryOutput],
) -> None:
    if output is None:
        return
    if not isinstance(output, PipelineParallelAuxiliaryOutput):
        raise RuntimeError(
            "auxiliary output does not support pipeline-parallel transport"
        )

    output_tensors = output.to_pp_tensors()
    if not output_tensors:
        raise RuntimeError("auxiliary PP output must contain at least one tensor")

    for name, tensor in output_tensors.items():
        if not isinstance(name, str) or not name:
            raise RuntimeError("auxiliary PP tensor names must be non-empty strings")
        if not torch.is_tensor(tensor):
            raise RuntimeError(f"auxiliary PP output {name!r} is not a tensor")
        key = f"{_OUTPUT_PREFIX}{name}"
        if key in tensors:
            raise RuntimeError(f"duplicate auxiliary PP tensor {name!r}")
        tensors[key] = tensor

View on GitHub (pinned to 0132848349)

Solutions

1. Return a PipelineParallelAuxiliaryOutput (subclass implementing to_pp_tensors/from it) from the model when PP is enabled
2. Skip attaching auxiliary outputs under PP until the type supports it
3. Fix the call site to not route non-PP outputs into the PP tensor dict

Example fix

# before
tensors['aux'] = plain_aux_output  # DeviceAuxiliaryOutput
# after
from sglang.srt.sampling... import PipelineParallelAuxiliaryOutput
class MyAuxOutput(PipelineParallelAuxiliaryOutput): ...
add_auxiliary_output_to_pp_tensors(tensors, MyAuxOutput(...))

from sglang.srt.sampling.sampling_observer_pp import PipelineParallelAuxiliaryOutput
if output is not None and not isinstance(output, PipelineParallelAuxiliaryOutput):
    output = None  # or convert/skip under PP

def is_pp_capable(o) -> bool:
    from sglang.srt.sampling.sampling_observer_pp import PipelineParallelAuxiliaryOutput
    return isinstance(o, PipelineParallelAuxiliaryOutput)

When it happens

Trigger: Calling add_auxiliary_output_to_pp_tensors with a plain DeviceAuxiliaryOutput (or subclass that never implemented PP transport) — typically a model returning auxiliary outputs under PP without PP support.

Common situations: Enabling pipeline parallelism with a model whose auxiliary outputs don't implement the PP protocol; new auxiliary output type added without PP subclass.

Related errors

• sampling observer does not support pipeline-parallel transpo
• Tag mismatch: expected CMD_LAYERWISE, got {payload.get('cmd'
• Tag mismatch: expected CMD_PUT_META, got {payload.get('cmd')
• Tag mismatch: expected CMD_STORE_COMPLETE, got {payload.get(
• auxiliary PP output must contain at least one tensor

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/eaa58909e86dab76. Report an issue: GitHub.