sgl-project/sglang · error · RuntimeError

sampling observer did not reconstruct its PP output

Error message

sampling observer did not reconstruct its PP output

What it means

observer.from_pp_tensors(output_tensors) returned None even though tensors were provided, so the output could not be reconstructed on the receiving stage.

Source

Thrown at python/sglang/srt/sampling/sampling_observer_pp.py:83

    output_tensors = {
        key.removeprefix(_OUTPUT_PREFIX): value
        for key, value in tensors.items()
        if key.startswith(_OUTPUT_PREFIX)
    }
    if not output_tensors:
        return None
    if observer is None:
        raise RuntimeError("received auxiliary PP output without a sampling observer")
    if not isinstance(observer, PipelineParallelSamplingObserver):
        raise RuntimeError(
            "sampling observer does not support pipeline-parallel transport"
        )
    if any(not torch.is_tensor(tensor) for tensor in output_tensors.values()):
        raise RuntimeError("received a non-tensor auxiliary PP output")

    output = observer.from_pp_tensors(output_tensors)
    if output is None:
        raise RuntimeError("sampling observer did not reconstruct its PP output")
    for name in output_tensors:
        del tensors[f"{_OUTPUT_PREFIX}{name}"]
    return output

View on GitHub (pinned to 0132848349)

Solutions

1. Make from_pp_tensors return a reconstructed output (or raise a descriptive error for missing fields)
2. Align field names between to_pp_tensors and from_pp_tensors, including any prefix handling
3. Add round-trip unit tests: from_pp_tensors(to_pp_tensors(x)) is not None

Example fix

# before
def from_pp_tensors(cls, tensors):
    if "missing_key" in tensors: return cls(...)
    return None
# after
def from_pp_tensors(cls, tensors):
    return cls(hidden=tensors["hidden"], logits=tensors["logits"])

# round-trip check
sample = output.to_pp_tensors()
assert type(output).from_pp_tensors(sample) is not None

When it happens

Trigger: A PipelineParallelSamplingObserver subclass whose from_pp_tensors returns None — e.g. expects a key that wasn't transported, or a stub/incomplete implementation.

Common situations: Observer expecting renamed/extra fields after a refactor while sender sends the old names; partial custom observer implementation.

Related errors

• received auxiliary PP output without a sampling observer
• sampling observer does not support pipeline-parallel transpo
• auxiliary output does not support pipeline-parallel transpor
• auxiliary PP output must contain at least one tensor
• auxiliary PP tensor names must be non-empty strings

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/51e0684f164fc587. Report an issue: GitHub.