sgl-project/sglang · error · RuntimeError

auxiliary PP output {name!r} is not a tensor

Error message

auxiliary PP output {name!r} is not a tensor

What it means

A value in the to_pp_tensors() mapping is not a torch.Tensor; PP transport only forwards tensors, so the payload is invalid.

Source

Thrown at python/sglang/srt/sampling/sampling_observer_pp.py:54

    tensors: MutableMapping[str, Any],
    output: Optional[DeviceAuxiliaryOutput],
) -> None:
    if output is None:
        return
    if not isinstance(output, PipelineParallelAuxiliaryOutput):
        raise RuntimeError(
            "auxiliary output does not support pipeline-parallel transport"
        )

    output_tensors = output.to_pp_tensors()
    if not output_tensors:
        raise RuntimeError("auxiliary PP output must contain at least one tensor")

    for name, tensor in output_tensors.items():
        if not isinstance(name, str) or not name:
            raise RuntimeError("auxiliary PP tensor names must be non-empty strings")
        if not torch.is_tensor(tensor):
            raise RuntimeError(f"auxiliary PP output {name!r} is not a tensor")
        key = f"{_OUTPUT_PREFIX}{name}"
        if key in tensors:
            raise RuntimeError(f"duplicate auxiliary PP tensor {name!r}")
        tensors[key] = tensor


def pop_auxiliary_output_from_pp_tensors(
    tensors: MutableMapping[str, Any],
    observer: Optional[SamplingObserver],
) -> Optional[DeviceAuxiliaryOutput]:
    output_tensors = {
        key.removeprefix(_OUTPUT_PREFIX): value
        for key, value in tensors.items()
        if key.startswith(_OUTPUT_PREFIX)
    }
    if not output_tensors:
        return None
    if observer is None:

View on GitHub (pinned to 0132848349)

Solutions

1. Convert all values to torch tensors in to_pp_tensors()
2. Pass non-tensor metadata through a separate channel

Example fix

# before
return {"logprob": float(v)}
# after
return {"logprob": torch.tensor(float(v))}

import torch
t = output.to_pp_tensors()
assert all(torch.is_tensor(v) for v in t.values())

import torch
def all_tensors(t: dict) -> bool:
    return all(torch.is_tensor(v) for v in t.values())

When it happens

Trigger: to_pp_tensors() returning numbers, lists, numpy arrays, or None among its values.

Common situations: Custom auxiliary output stuffing scalars/arrays into the tensor dict instead of tensorizing them.

Related errors

• received a non-tensor auxiliary PP output
• auxiliary PP output must contain at least one tensor
• auxiliary PP tensor names must be non-empty strings
• duplicate auxiliary PP tensor {name!r}
• kv-canary: {name} must be on {reference_name}'s device {refe

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7240a20a547cc61c. Report an issue: GitHub.