sgl-project/sglang · error · RuntimeError

sampling observer does not support pipeline-parallel transpo

Error message

sampling observer does not support pipeline-parallel transport

What it means

Auxiliary PP tensors were received and an observer exists, but it is not a PipelineParallelSamplingObserver, so it lacks from_pp_tensors to reconstruct the output.

Source

Thrown at python/sglang/srt/sampling/sampling_observer_pp.py:75

            raise RuntimeError(f"duplicate auxiliary PP tensor {name!r}")
        tensors[key] = tensor


def pop_auxiliary_output_from_pp_tensors(
    tensors: MutableMapping[str, Any],
    observer: Optional[SamplingObserver],
) -> Optional[DeviceAuxiliaryOutput]:
    output_tensors = {
        key.removeprefix(_OUTPUT_PREFIX): value
        for key, value in tensors.items()
        if key.startswith(_OUTPUT_PREFIX)
    }
    if not output_tensors:
        return None
    if observer is None:
        raise RuntimeError("received auxiliary PP output without a sampling observer")
    if not isinstance(observer, PipelineParallelSamplingObserver):
        raise RuntimeError(
            "sampling observer does not support pipeline-parallel transport"
        )
    if any(not torch.is_tensor(tensor) for tensor in output_tensors.values()):
        raise RuntimeError("received a non-tensor auxiliary PP output")

    output = observer.from_pp_tensors(output_tensors)
    if output is None:
        raise RuntimeError("sampling observer did not reconstruct its PP output")
    for name in output_tensors:
        del tensors[f"{_OUTPUT_PREFIX}{name}"]
    return output

View on GitHub (pinned to 0132848349)

Solutions

1. Use/wrap a PipelineParallelSamplingObserver on PP receiving stages
2. Implement from_pp_tensors on the observer subclass
3. Disable auxiliary PP outputs if the observer can't consume them

from sglang.srt.sampling.sampling_observer_pp import PipelineParallelSamplingObserver
if aux_present and not isinstance(observer, PipelineParallelSamplingObserver):
    raise/convert before popping

def is_pp_observer(o) -> bool:
    from sglang.srt.sampling.sampling_observer_pp import PipelineParallelSamplingObserver
    return isinstance(o, PipelineParallelSamplingObserver)

When it happens

Trigger: Passing a plain SamplingObserver (or subclass without PP support) to pop_auxiliary_output_from_pp_tensors.

Common situations: Reusing a non-PP observer implementation under pipeline parallelism; new observer type not implementing the PP protocol.

Related errors

• auxiliary output does not support pipeline-parallel transpor
• received auxiliary PP output without a sampling observer
• sampling observer did not reconstruct its PP output
• auxiliary PP output must contain at least one tensor
• auxiliary PP tensor names must be non-empty strings

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/af76aa0ff10d0648. Report an issue: GitHub.