sgl-project/sglang · error · RuntimeError

received a non-tensor auxiliary PP output

Error message

received a non-tensor auxiliary PP output

What it means

Defensive check on the receiving side: at least one value in the received auxiliary PP tensors is not a torch.Tensor, which shouldn't happen if the sender-side validation ran.

Source

Thrown at python/sglang/srt/sampling/sampling_observer_pp.py:79

def pop_auxiliary_output_from_pp_tensors(
    tensors: MutableMapping[str, Any],
    observer: Optional[SamplingObserver],
) -> Optional[DeviceAuxiliaryOutput]:
    output_tensors = {
        key.removeprefix(_OUTPUT_PREFIX): value
        for key, value in tensors.items()
        if key.startswith(_OUTPUT_PREFIX)
    }
    if not output_tensors:
        return None
    if observer is None:
        raise RuntimeError("received auxiliary PP output without a sampling observer")
    if not isinstance(observer, PipelineParallelSamplingObserver):
        raise RuntimeError(
            "sampling observer does not support pipeline-parallel transport"
        )
    if any(not torch.is_tensor(tensor) for tensor in output_tensors.values()):
        raise RuntimeError("received a non-tensor auxiliary PP output")

    output = observer.from_pp_tensors(output_tensors)
    if output is None:
        raise RuntimeError("sampling observer did not reconstruct its PP output")
    for name in output_tensors:
        del tensors[f"{_OUTPUT_PREFIX}{name}"]
    return output

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure all stages run the same sglang version with sender-side validation
2. Avoid mutating the tensor dict between add_ and pop_ calls
3. Tensorize any custom injected values

import torch
assert all(torch.is_tensor(v) for v in output_tensors.values()), "non-tensor leaked into PP tensors"

When it happens

Trigger: Tensors mutated/replaced with non-tensors between add and pop — e.g. custom transport code, pickling artifacts, or a tensor dict polluted by other code paths.

Common situations: Custom inter-process tensor transfer converting values; monkeypatched or older-version sender missing validation (version skew between stages).

Related errors

• auxiliary PP output {name!r} is not a tensor
• auxiliary PP output must contain at least one tensor
• auxiliary PP tensor names must be non-empty strings
• duplicate auxiliary PP tensor {name!r}
• kv-canary: {name} must be on {reference_name}'s device {refe

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/50f15bd6db1f5302. Report an issue: GitHub.