sgl-project/sglang · error · RuntimeError

sampling observer did not reconstruct its PP output

Error message

sampling observer did not reconstruct its PP output

What it means

After receiving the auxiliary PP payload, the sampling observer failed to reconstruct its expected output structure (e.g. missing/extra tensors relative to what it registered). Indicates producer/consumer asymmetry in the auxiliary tensor set.

Solutions

1. Make from_pp_tensors return a reconstructed output (or raise a descriptive error for missing fields)
2. Align field names between to_pp_tensors and from_pp_tensors, including any prefix handling
3. Add round-trip unit tests: from_pp_tensors(to_pp_tensors(x)) is not None

Example fix

# before
def from_pp_tensors(cls, tensors):
    if "missing_key" in tensors: return cls(...)
    return None
# after
def from_pp_tensors(cls, tensors):
    return cls(hidden=tensors["hidden"], logits=tensors["logits"])

# round-trip check
sample = output.to_pp_tensors()
assert type(output).from_pp_tensors(sample) is not None

When it happens

Trigger: A PipelineParallelSamplingObserver subclass whose from_pp_tensors returns None — e.g. expects a key that wasn't transported, or a stub/incomplete implementation.

Common situations: Observer expecting renamed/extra fields after a refactor while sender sends the old names; partial custom observer implementation.

Related errors

• received auxiliary PP output without a sampling observer
• sampling observer does not support pipeline-parallel…
• auxiliary output does not support pipeline-parallel…
• auxiliary PP output must contain at least one tensor
• auxiliary PP output is not a tensor

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/51e0684f164fc587. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/sampling/sampling_observer_pp.py:83

    output_tensors = {
        key.removeprefix(_OUTPUT_PREFIX): value
        for key, value in tensors.items()
        if key.startswith(_OUTPUT_PREFIX)
    }
    if not output_tensors:
        return None
    if observer is None:
        raise RuntimeError("received auxiliary PP output without a sampling observer")
    if not isinstance(observer, PipelineParallelSamplingObserver):
        raise RuntimeError(
            "sampling observer does not support pipeline-parallel transport"
        )
    if any(not torch.is_tensor(tensor) for tensor in output_tensors.values()):
        raise RuntimeError("received a non-tensor auxiliary PP output")

    output = observer.from_pp_tensors(output_tensors)
    if output is None:
        raise RuntimeError("sampling observer did not reconstruct its PP output")
    for name in output_tensors:
        del tensors[f"{_OUTPUT_PREFIX}{name}"]
    return output

View on GitHub (pinned to 0132848349)