{"record":{"id":"4b331a4daff70615","repo":"apache/beam","slug":"error-decoding-residual-header","errorCode":null,"errorMessage":"error decoding residual header:","messagePattern":"error decoding residual header:","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"sdks/go/pkg/beam/runners/prism/internal/engine/elementmanager.go","lineNumber":828,"sourceCode":"\t\t\tdelayed = map[mtime.Time][]Residual{}\n\t\t}\n\t\tfireAt := emNow.Add(r.Delay)\n\t\tdelayed[fireAt] = append(delayed[fireAt], r)\n\t}\n\treturn immediate, delayed\n}\n\nfunc reElementResiduals(residuals []Residual, inputInfo PColInfo, rb RunBundle) []element {\n\tvar unprocessedElements []element\n\tfor _, residual := range residuals {\n\t\tbuf := bytes.NewBuffer(residual.Element)\n\t\tws, et, pn, err := exec.DecodeWindowedValueHeader(inputInfo.WDec, buf)\n\t\tif err != nil {\n\t\t\tif err == io.EOF {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\tslog.Error(\"reElementResiduals: error decoding residual header\", \"error\", err, \"bundle\", rb)\n\t\t\tpanic(\"error decoding residual header:\" + err.Error())\n\t\t}\n\t\tif len(ws) == 0 {\n\t\t\tslog.Warn(\"reElementResiduals: sdk provided a windowed value header 0 windows\", \"bundle\", rb)\n\t\t}\n\t\t// POSSIBLY BAD PATTERN: The buffer is invalidated on the next call, which doesn't always happen.\n\t\t// But the decoder won't be mutating the buffer bytes, just reading the data. So the elmBytes\n\t\t// should remain pointing to the whole element, and we should have a copy of the key bytes.\n\t\t// Ideally, we're simply refering to the key part of the existing buffer.\n\t\telmBytes := buf.Bytes()\n\t\tvar keyBytes []byte\n\t\tif inputInfo.KeyDec != nil {\n\t\t\tkeyBytes = inputInfo.KeyDec(buf)\n\t\t}\n\n\t\tfor _, w := range ws {\n\t\t\tunprocessedElements = append(unprocessedElements,\n\t\t\t\telement{\n\t\t\t\t\twindow:    w,","sourceCodeStart":810,"sourceCodeEnd":846,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/go/pkg/beam/runners/prism/internal/engine/elementmanager.go#L810-L846","documentation":"When re-materializing residual elements of an unfinished bundle, reElementResiduals decodes a windowed-value header from the residual buffer. A non-EOF decode error means the residual bytes are malformed or produced by an incompatible coder, so prism panics after logging.","triggerScenarios":"DecodeWindowedValueHeader returns an error other than io.EOF while decoding a residual bundle's buffer — truncated residual data, coder mismatch between the writer and reader of the residual split, or corrupt on-buffer content.","commonSituations":"Bundle splitting producing residuals the decoder can't parse; SDK/runner coder version skew; potential prism bug in residual buffer management (noted buffer-invalidation caveat in code).","solutions":["Retry the pipeline; if intermittent, report as a possible prism residual-buffer bug","Upgrade Beam to the latest release where residual handling fixes may land","Disable/reduce bundle splitting (e.g. via pipeline options) as a workaround to avoid residual paths","Capture the logged 'error' field and bundle details to include in a bug report"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// pre-check residual buffer plausibility before reprocessing\nif len(buf.Bytes()) == 0 { return errors.New(\"residual buffer empty, skipping re-element\") }","typeGuard":null,"tryCatchPattern":"func safeReElement(rb Bundle, buf []byte) (ok bool) {\n  defer func() { if r := recover(); r != nil { log.Errorf(\"residual decode panicked: %v\", r); ok = false } }()\n  reElementResiduals(rb, buf); return true\n}","preventionTips":["Keep Beam versions consistent across SDK and runner","Reduce reliance on aggressive bundle splitting if residuals fail intermittently","Capture and attach prism logs when filing residual-decode bugs"],"tags":["go","beam","prism","residual","bundle-splitting","panic"],"backgroundTag":"internal-invariant-violation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}