sgl-project/sglang · error · RuntimeError

auxiliary PP output must contain at least one tensor

Error message

auxiliary PP output must contain at least one tensor

What it means

A PipelineParallelAuxiliaryOutput serialized to zero tensors, so there is nothing to forward across pipeline stages and the transport would be a no-op the receiver can't reconstruct.

Source

Thrown at python/sglang/srt/sampling/sampling_observer_pp.py:48

_OUTPUT_PREFIX = "__sampling_observer_output__."


def add_auxiliary_output_to_pp_tensors(
    tensors: MutableMapping[str, Any],
    output: Optional[DeviceAuxiliaryOutput],
) -> None:
    if output is None:
        return
    if not isinstance(output, PipelineParallelAuxiliaryOutput):
        raise RuntimeError(
            "auxiliary output does not support pipeline-parallel transport"
        )

    output_tensors = output.to_pp_tensors()
    if not output_tensors:
        raise RuntimeError("auxiliary PP output must contain at least one tensor")

    for name, tensor in output_tensors.items():
        if not isinstance(name, str) or not name:
            raise RuntimeError("auxiliary PP tensor names must be non-empty strings")
        if not torch.is_tensor(tensor):
            raise RuntimeError(f"auxiliary PP output {name!r} is not a tensor")
        key = f"{_OUTPUT_PREFIX}{name}"
        if key in tensors:
            raise RuntimeError(f"duplicate auxiliary PP tensor {name!r}")
        tensors[key] = tensor


def pop_auxiliary_output_from_pp_tensors(
    tensors: MutableMapping[str, Any],
    observer: Optional[SamplingObserver],
) -> Optional[DeviceAuxiliaryOutput]:
    output_tensors = {
        key.removeprefix(_OUTPUT_PREFIX): value

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure to_pp_tensors() returns at least one tensor, or skip calling add when empty
2. Make from_pp_tensors consistent with the non-empty requirement
3. Add a guard in the caller: only attach when to_pp_tensors() is truthy

Example fix

# before
if output is not None:
    add_auxiliary_output_to_pp_tensors(tensors, output)
# after
if output is not None and output.to_pp_tensors():
    add_auxiliary_output_to_pp_tensors(tensors, output)

if output is not None and not output.to_pp_tensors():
    output = None  # nothing to forward

When it happens

Trigger: An auxiliary output whose to_pp_tensors() returns {} or an empty mapping passed into add_auxiliary_output_to_pp_tensors.

Common situations: Custom auxiliary output where all fields are None/optional and to_pp_tensors filters them all out; stub implementation returning {}.

Related errors

• auxiliary PP tensor names must be non-empty strings
• auxiliary PP output {name!r} is not a tensor
• duplicate auxiliary PP tensor {name!r}
• received a non-tensor auxiliary PP output
• kv-canary: RealKvSource.read_bytes must be a positive multip

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a9e69df6cf0f42b0. Report an issue: GitHub.