sgl-project/sglang · error · RuntimeError

received auxiliary PP output without a sampling observer

Error message

received auxiliary PP output without a sampling observer

What it means

On the receiving pipeline stage, auxiliary PP tensors arrived in the transported dict but no SamplingObserver was provided to reconstruct them.

Source

Thrown at python/sglang/srt/sampling/sampling_observer_pp.py:73

        key = f"{_OUTPUT_PREFIX}{name}"
        if key in tensors:
            raise RuntimeError(f"duplicate auxiliary PP tensor {name!r}")
        tensors[key] = tensor


def pop_auxiliary_output_from_pp_tensors(
    tensors: MutableMapping[str, Any],
    observer: Optional[SamplingObserver],
) -> Optional[DeviceAuxiliaryOutput]:
    output_tensors = {
        key.removeprefix(_OUTPUT_PREFIX): value
        for key, value in tensors.items()
        if key.startswith(_OUTPUT_PREFIX)
    }
    if not output_tensors:
        return None
    if observer is None:
        raise RuntimeError("received auxiliary PP output without a sampling observer")
    if not isinstance(observer, PipelineParallelSamplingObserver):
        raise RuntimeError(
            "sampling observer does not support pipeline-parallel transport"
        )
    if any(not torch.is_tensor(tensor) for tensor in output_tensors.values()):
        raise RuntimeError("received a non-tensor auxiliary PP output")

    output = observer.from_pp_tensors(output_tensors)
    if output is None:
        raise RuntimeError("sampling observer did not reconstruct its PP output")
    for name in output_tensors:
        del tensors[f"{_OUTPUT_PREFIX}{name}"]
    return output

View on GitHub (pinned to 0132848349)

Solutions

1. Construct/pass a PipelineParallelSamplingObserver on the stage that pops the outputs
2. Disable auxiliary output emission on earlier stages when no observer is configured
3. Ensure observer wiring is propagated with the batch result across stages

from sglang.srt.sampling.sampling_observer_pp import pop_auxiliary_output_from_pp_tensors, _OUTPUT_PREFIX
has_aux = any(k.startswith(_OUTPUT_PREFIX) for k in tensors)
if has_aux:
    assert observer is not None, "configure a sampling observer on the final PP stage"

When it happens

Trigger: pop_auxiliary_output_from_pp_tensors finding prefix-keys while observer is None — last PP stage lacking a sampling observer configuration while earlier stages emitted auxiliary outputs.

Common situations: PP deployment where the final stage has sampling observers disabled (e.g. no sampling params requiring observation) but the model still emits auxiliary outputs.

Related errors

• sampling observer does not support pipeline-parallel transpo
• sampling observer did not reconstruct its PP output
• auxiliary output does not support pipeline-parallel transpor
• auxiliary PP output must contain at least one tensor
• auxiliary PP tensor names must be non-empty strings

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0f6c2e5d6c51fec0. Report an issue: GitHub.