sgl-project/sglang · error · RuntimeError

auxiliary PP tensor names must be non-empty strings

Error message

auxiliary PP tensor names must be non-empty strings

What it means

A key in the to_pp_tensors() mapping is not a non-empty string, so it cannot be namespaced into the PP tensor dict with the output prefix.

Source

Thrown at python/sglang/srt/sampling/sampling_observer_pp.py:52

def add_auxiliary_output_to_pp_tensors(
    tensors: MutableMapping[str, Any],
    output: Optional[DeviceAuxiliaryOutput],
) -> None:
    if output is None:
        return
    if not isinstance(output, PipelineParallelAuxiliaryOutput):
        raise RuntimeError(
            "auxiliary output does not support pipeline-parallel transport"
        )

    output_tensors = output.to_pp_tensors()
    if not output_tensors:
        raise RuntimeError("auxiliary PP output must contain at least one tensor")

    for name, tensor in output_tensors.items():
        if not isinstance(name, str) or not name:
            raise RuntimeError("auxiliary PP tensor names must be non-empty strings")
        if not torch.is_tensor(tensor):
            raise RuntimeError(f"auxiliary PP output {name!r} is not a tensor")
        key = f"{_OUTPUT_PREFIX}{name}"
        if key in tensors:
            raise RuntimeError(f"duplicate auxiliary PP tensor {name!r}")
        tensors[key] = tensor


def pop_auxiliary_output_from_pp_tensors(
    tensors: MutableMapping[str, Any],
    observer: Optional[SamplingObserver],
) -> Optional[DeviceAuxiliaryOutput]:
    output_tensors = {
        key.removeprefix(_OUTPUT_PREFIX): value
        for key, value in tensors.items()
        if key.startswith(_OUTPUT_PREFIX)
    }
    if not output_tensors:

View on GitHub (pinned to 0132848349)

Solutions

1. Make to_pp_tensors() return Mapping[str, torch.Tensor] with non-empty string keys
2. Add a unit test asserting all keys are non-empty strings

Example fix

# before
def to_pp_tensors(self):
    return {0: self.hidden}
# after
def to_pp_tensors(self):
    return {"hidden": self.hidden}

t = output.to_pp_tensors()
assert all(isinstance(k, str) and k for k in t)

def valid_pp_keys(t: dict) -> bool:
    return all(isinstance(k, str) and k for k in t)

When it happens

Trigger: An auxiliary output whose to_pp_tensors() returns a dict with None, empty, or non-str keys (e.g. ints or enum keys).

Common situations: Custom implementation using enum/int keys or tuple keys in the tensor mapping.

Related errors

• auxiliary PP output must contain at least one tensor
• auxiliary PP output {name!r} is not a tensor
• duplicate auxiliary PP tensor {name!r}
• received a non-tensor auxiliary PP output
• kv-canary: RealKvSource.read_bytes must be a positive multip

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a2a0e361cea5a0c7. Report an issue: GitHub.