{"record":{"id":"eaa58909e86dab76","repo":"sgl-project/sglang","slug":"auxiliary-output-does-not-support-pipeline-paralle","errorCode":null,"errorMessage":"auxiliary output does not support pipeline-parallel transport","messagePattern":"auxiliary output does not support pipeline-parallel transport","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/sampling/sampling_observer_pp.py","lineNumber":42,"sourceCode":"\n@runtime_checkable\nclass PipelineParallelSamplingObserver(Protocol):\n    def from_pp_tensors(\n        self, tensors: Mapping[str, torch.Tensor]\n    ) -> DeviceAuxiliaryOutput: ...\n\n\n_OUTPUT_PREFIX = \"__sampling_observer_output__.\"\n\n\ndef add_auxiliary_output_to_pp_tensors(\n    tensors: MutableMapping[str, Any],\n    output: Optional[DeviceAuxiliaryOutput],\n) -> None:\n    if output is None:\n        return\n    if not isinstance(output, PipelineParallelAuxiliaryOutput):\n        raise RuntimeError(\n            \"auxiliary output does not support pipeline-parallel transport\"\n        )\n\n    output_tensors = output.to_pp_tensors()\n    if not output_tensors:\n        raise RuntimeError(\"auxiliary PP output must contain at least one tensor\")\n\n    for name, tensor in output_tensors.items():\n        if not isinstance(name, str) or not name:\n            raise RuntimeError(\"auxiliary PP tensor names must be non-empty strings\")\n        if not torch.is_tensor(tensor):\n            raise RuntimeError(f\"auxiliary PP output {name!r} is not a tensor\")\n        key = f\"{_OUTPUT_PREFIX}{name}\"\n        if key in tensors:\n            raise RuntimeError(f\"duplicate auxiliary PP tensor {name!r}\")\n        tensors[key] = tensor\n\n","sourceCodeStart":24,"sourceCodeEnd":60,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/sampling/sampling_observer_pp.py#L24-L60","documentation":"During pipeline-parallel sampling, a non-PP auxiliary output object was passed to the PP tensor transport helper; only PipelineParallelAuxiliaryOutput instances implement to_pp_tensors.","triggerScenarios":"Calling add_auxiliary_output_to_pp_tensors with a plain DeviceAuxiliaryOutput (or subclass that never implemented PP transport) — typically a model returning auxiliary outputs under PP without PP support.","commonSituations":"Enabling pipeline parallelism with a model whose auxiliary outputs don't implement the PP protocol; new auxiliary output type added without PP subclass.","solutions":["Return a PipelineParallelAuxiliaryOutput (subclass implementing to_pp_tensors/from it) from the model when PP is enabled","Skip attaching auxiliary outputs under PP until the type supports it","Fix the call site to not route non-PP outputs into the PP tensor dict"],"exampleFix":"# before\ntensors['aux'] = plain_aux_output  # DeviceAuxiliaryOutput\n# after\nfrom sglang.srt.sampling... import PipelineParallelAuxiliaryOutput\nclass MyAuxOutput(PipelineParallelAuxiliaryOutput): ...\nadd_auxiliary_output_to_pp_tensors(tensors, MyAuxOutput(...))","handlingStrategy":"type-guard","validationCode":"from sglang.srt.sampling.sampling_observer_pp import PipelineParallelAuxiliaryOutput\nif output is not None and not isinstance(output, PipelineParallelAuxiliaryOutput):\n    output = None  # or convert/skip under PP","typeGuard":"def is_pp_capable(o) -> bool:\n    from sglang.srt.sampling.sampling_observer_pp import PipelineParallelAuxiliaryOutput\n    return isinstance(o, PipelineParallelAuxiliaryOutput)","tryCatchPattern":null,"preventionTips":["Implement PP transport on every auxiliary output type used with pipeline parallelism","Gate auxiliary outputs on is_pp_capable before attaching"],"tags":["pipeline-parallel","sampling","type-validation","distributed"],"backgroundTag":"protocol-mismatch-distributed-transport","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}