{"record":{"id":"51e0684f164fc587","repo":"sgl-project/sglang","slug":"sampling-observer-did-not-reconstruct-its-pp-outpu","errorCode":null,"errorMessage":"sampling observer did not reconstruct its PP output","messagePattern":"sampling observer did not reconstruct its PP output","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/sampling/sampling_observer_pp.py","lineNumber":83,"sourceCode":"    output_tensors = {\n        key.removeprefix(_OUTPUT_PREFIX): value\n        for key, value in tensors.items()\n        if key.startswith(_OUTPUT_PREFIX)\n    }\n    if not output_tensors:\n        return None\n    if observer is None:\n        raise RuntimeError(\"received auxiliary PP output without a sampling observer\")\n    if not isinstance(observer, PipelineParallelSamplingObserver):\n        raise RuntimeError(\n            \"sampling observer does not support pipeline-parallel transport\"\n        )\n    if any(not torch.is_tensor(tensor) for tensor in output_tensors.values()):\n        raise RuntimeError(\"received a non-tensor auxiliary PP output\")\n\n    output = observer.from_pp_tensors(output_tensors)\n    if output is None:\n        raise RuntimeError(\"sampling observer did not reconstruct its PP output\")\n    for name in output_tensors:\n        del tensors[f\"{_OUTPUT_PREFIX}{name}\"]\n    return output\n","sourceCodeStart":65,"sourceCodeEnd":87,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/sampling/sampling_observer_pp.py#L65-L87","documentation":"observer.from_pp_tensors(output_tensors) returned None even though tensors were provided, so the output could not be reconstructed on the receiving stage.","triggerScenarios":"A PipelineParallelSamplingObserver subclass whose from_pp_tensors returns None — e.g. expects a key that wasn't transported, or a stub/incomplete implementation.","commonSituations":"Observer expecting renamed/extra fields after a refactor while sender sends the old names; partial custom observer implementation.","solutions":["Make from_pp_tensors return a reconstructed output (or raise a descriptive error for missing fields)","Align field names between to_pp_tensors and from_pp_tensors, including any prefix handling","Add round-trip unit tests: from_pp_tensors(to_pp_tensors(x)) is not None"],"exampleFix":"# before\ndef from_pp_tensors(cls, tensors):\n    if \"missing_key\" in tensors: return cls(...)\n    return None\n# after\ndef from_pp_tensors(cls, tensors):\n    return cls(hidden=tensors[\"hidden\"], logits=tensors[\"logits\"])","handlingStrategy":"validation","validationCode":"# round-trip check\nsample = output.to_pp_tensors()\nassert type(output).from_pp_tensors(sample) is not None","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep to_pp_tensors/from_pp_tensors field names in sync","Add round-trip unit tests for every observer implementation"],"tags":["pipeline-parallel","sampling","observer","reconstruction"],"backgroundTag":"deserialization-returned-null","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}