{"record":{"id":"aecf331b46447624","repo":"apache/beam","slug":"decodemultichunkstream-chunk-size-decoding-failed","errorCode":null,"errorMessage":"decodeMultiChunkStream chunk size decoding failed","messagePattern":"decodeMultiChunkStream chunk size decoding failed","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"sdks/go/pkg/beam/core/runtime/exec/fullvalue.go","lineNumber":351,"sourceCode":"\t// If we have a stream already, use that.\n\tif s.stream != nil {\n\t\treturn s.stream.Read()\n\t}\n\n\t// If our next value is at the chunk size, then re-set the chunk and size.\n\tif s.next == s.chunk {\n\t\ts.chunk = 0\n\t\ts.next = 0\n\t}\n\n\t// We're at the start of a chunk, see if there's a next chunk.\n\tif s.chunk == 0 && s.next == 0 {\n\t\tchunk, err := coder.DecodeVarInt(s.r.reader)\n\t\tif err != nil {\n\t\t\tif err == io.EOF {\n\t\t\t\treturn nil, io.EOF\n\t\t\t}\n\t\t\treturn nil, errors.Wrap(err, \"decodeMultiChunkStream chunk size decoding failed\")\n\t\t}\n\t\ts.chunk = chunk\n\t}\n\tswitch {\n\tcase s.chunk == 0:\n\t\t// If the chunk is still 0, then we're done.\n\t\ts.r = nil\n\t\ts.d = nil\n\t\ts.ret = FullValue{}\n\t\treturn nil, io.EOF\n\tcase s.chunk > 0:\n\t\terr := s.d.DecodeTo(s.r, &s.ret)\n\t\tif err != nil {\n\t\t\treturn nil, errors.Wrap(err, \"decodeStream value decode failed\")\n\t\t}\n\t\ts.next++\n\t\treturn &s.ret, nil\n\tcase s.chunk == -1:","sourceCodeStart":333,"sourceCodeEnd":369,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/go/pkg/beam/core/runtime/exec/fullvalue.go#L333-L369","documentation":"The multi-chunk decode stream reads a varint chunk size before each chunk; failure to decode that size prefix (other than clean EOF) is wrapped as 'decodeMultiChunkStream chunk size decoding failed'. It indicates the stream is malformed or cut off exactly where a chunk-length header was expected.","triggerScenarios":"Read() on decodeMultiChunkStream when s.chunk == 0 && s.next == 0 and coder.DecodeVarInt(s.r.reader) returns a non-EOF error: the underlying reader produced invalid bytes or the connection died mid-header.","commonSituations":"Network interruption between runners killing the gRPC data stream; desynchronized reader after an earlier partial decode (custom coder read the wrong number of bytes); runner-issued payload truncation.","solutions":["Rerun/retry the bundle — the runner will restart the stream from the sender.","Check earlier errors in the log; a prior custom-coder misread can desynchronize the stream.","Verify stable connectivity between runner and workers (Dataflow/Spark/Flink data plane health)."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"if errors.Is(err, io.EOF) { /* clean end */ } else if strings.Contains(err.Error(), \"chunk size decoding failed\") {\n    return retryable // stream desync/corruption, runner should re-deliver\n}","preventionTips":["Ensure custom coders read exactly the bytes they wrote to avoid stream desync.","Maintain stable network paths between runner and workers.","Rerun failures once before deep debugging."],"tags":["go","apache-beam","decoding","stream"],"backgroundTag":"stream-decode-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}