{"record":{"id":"83b755fe7dc0fc14","repo":"apache/beam","slug":"number-of-fields-is-less-than-byte-array-v-v","errorCode":null,"errorMessage":"number of fields is less than byte array %v < %v","messagePattern":"number of fields is less than byte array (.+?) < (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"sdks/go/pkg/beam/core/graph/coder/row.go","lineNumber":175,"sourceCode":"// examined once during decoding using the IsFieldNil helper function.\n//\n// If there are no nil fields encoded,the byte array will be nil, and no\n// encoded fields will be nil.\nfunc ReadRowHeader(r io.Reader) (int, []byte, error) {\n\tnf, err := DecodeVarInt(r) // is for checksum purposes (old vs new versions of a schemas)\n\tif err != nil {\n\t\treturn 0, nil, err\n\t}\n\tl, err := DecodeVarInt(r) // read the length prefix for the packed bits.\n\tif err != nil {\n\t\treturn int(nf), nil, err\n\t}\n\tif l == 0 {\n\t\t// A zero length byte array means no nils.\n\t\treturn int(nf), nil, nil\n\t}\n\tif nf < l {\n\t\treturn int(nf), nil, fmt.Errorf(\"number of fields is less than byte array %v < %v\", nf, l)\n\t}\n\tnils := make([]byte, l)\n\tif err := ioutilx.ReadNBufUnsafe(r, nils); err != nil {\n\t\treturn int(nf), nil, err\n\t}\n\treturn int(nf), nils, nil\n}\n\n// IsFieldNil examines the passed in packed bits nils buffer\n// and returns true if the field at that index wasn't encoded\n// and can be skipped in decoding.\nfunc IsFieldNil(nils []byte, f int) bool {\n\ti, b := f/8, f%8\n\t// https://github.com/apache/beam/issues/21232: The row header can elide trailing 0 bytes,\n\t// and we shouldn't care if there are trailing 0 bytes when doing a lookup.\n\treturn i < len(nils) && len(nils) != 0 && (nils[i]>>uint8(b))&0x1 == 1\n}\n","sourceCodeStart":157,"sourceCodeEnd":193,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/go/pkg/beam/core/graph/coder/row.go#L157-L193","documentation":"ReadRowHeader decodes a Beam Schema Row header: after reading the field count nf and the nils bit-field byte-array length l, it validates nf >= l because each byte in the nils array can represent at most one field's nil-ness. This error means the header claims more nil-bit bytes than there are fields, so the encoded row is corrupt or was written by an incompatible encoder.","triggerScenarios":"Decoding a stream with row coder ReadRowHeader where the nils-length varint exceeds the preceding field count — typically from data corruption, wrong coder being used to decode, or bytes written by a different Beam version/protocol.","commonSituations":"Mixing Beam SDK versions between writer and reader; a pipeline deserializing data with the wrong coder assigned; truncated/corrupted serialized data shifting varint boundaries.","solutions":["Verify the same row coder and schema were used on both encode and decode sides.","Check for Beam SDK version mismatch between the job that wrote the data and the one reading it.","Validate/re-serialize the source data; inspect raw bytes around the header for corruption.","Ensure decoded byte offsets are aligned (a prior decode bug can leave the reader mid-record)."],"exampleFix":"// before\nrowDecoder.Decode(r, typex.NewRowType(wrongSchema, nil)) // schema mismatch\n// after\nrowDecoder.Decode(r, typex.NewRowType(originalSchema, nil)) // use the schema the data was written with","handlingStrategy":"validation","validationCode":"if l > nf { return fmt.Errorf(\"corrupt row header: nils bytes %d exceed fields %d\", l, nf) }","typeGuard":null,"tryCatchPattern":"nf, nils, err := coder.ReadRowHeader(r)\nif err != nil && strings.Contains(err.Error(), \"number of fields is less than byte array\") {\n\treturn fmt.Errorf(\"data written with incompatible coder/schema: %w\", err)\n}","preventionTips":["Use the same row coder and schema on encode and decode sides.","Pin Beam SDK versions across services sharing serialized row data.","Verify data integrity (checksums) for serialized row files."],"tags":["go","beam","serialization","row-coder","corruption"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}