{"record":{"id":"4697c8a5e8e17809","repo":"apache/beam","slug":"invalid-varint-in-ordered-list-entry","errorCode":null,"errorMessage":"invalid varint in ordered list entry","messagePattern":"invalid varint in ordered list entry","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"sdks/go/pkg/beam/core/runtime/exec/userstate.go","lineNumber":607,"sourceCode":"\t// Read varint byte-by-byte.\n\tvar buf [10]byte // max varint size\n\tvar n int\n\tfor n = 0; n < len(buf); n++ {\n\t\t_, err := r.Read(buf[n : n+1])\n\t\tif err != nil {\n\t\t\tif n == 0 {\n\t\t\t\treturn state.OrderedListEntry{}, err\n\t\t\t}\n\t\t\treturn state.OrderedListEntry{}, fmt.Errorf(\"unexpected error reading varint: %w\", err)\n\t\t}\n\t\tif buf[n]&0x80 == 0 {\n\t\t\tn++\n\t\t\tbreak\n\t\t}\n\t}\n\tsortKey, consumed := protowire.ConsumeVarint(buf[:n])\n\tif consumed < 0 {\n\t\treturn state.OrderedListEntry{}, fmt.Errorf(\"invalid varint in ordered list entry\")\n\t}\n\n\tdec := MakeElementDecoder(coder.SkipW(c))\n\tfv, err := dec.Decode(r)\n\tif err != nil {\n\t\treturn state.OrderedListEntry{}, err\n\t}\n\treturn state.OrderedListEntry{SortKey: int64(sortKey), Value: fv.Elm}, nil\n}\n\nfunc (s *stateProvider) encodeKey(userStateID string, key any) ([]byte, error) {\n\tfv := FullValue{Elm: key}\n\tenc := MakeElementEncoder(coder.SkipW(s.keyCodersByID[userStateID]))\n\tvar b bytes.Buffer\n\terr := enc.Encode(&fv, &b)\n\tif err != nil {\n\t\treturn nil, err\n\t}","sourceCodeStart":589,"sourceCodeEnd":625,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/go/pkg/beam/core/runtime/exec/userstate.go#L589-L625","documentation":"After reading up to 10 varint bytes for an ordered list entry's sort key, decodeOrderedListEntry validates the buffer with protowire.ConsumeVarint. If the consumed length is negative, the bytes don't form a valid protobuf varint (e.g. truncated 10-byte continuation). The entry is rejected rather than mis-decoded.","triggerScenarios":"Reading an ordered list state entry whose sort-key bytes are malformed — buffer overflowed the 10-byte loop without a terminating byte (0x80 bit clear), or the bytes are corrupt.","commonSituations":"State data written by a different SDK version or coder layout; corrupted/truncated state payloads; a varint exceeding the maximum 10-byte protobuf encoding; bit-level bugs in custom state serialization.","solutions":["Verify the state was written by a compatible Beam version using the same varint/coder conventions.","Clear/rebuild the affected state data if it came from an older or corrupted job.","Check the producer of the sort key — oversized or non-canonical varints (>10 bytes) are invalid in protobuf.","Add validation at write time so sort keys are encoded via protowire.AppendVarint."],"exampleFix":"// before: raw manual encoding of sort key\nbuf = append(buf, byte(sortKey)) // breaks for large keys\n// after\nbuf = protowire.AppendVarint(buf, uint64(sortKey)) // canonical varint","handlingStrategy":"validation","validationCode":"if len(buf) == 0 || len(buf) > 10 { return errors.New(\"sort key varint out of protobuf bounds\") }\nif n := protowire.ConsumeVarint(buf); n < 0 { return errors.New(\"invalid varint bytes\") }","typeGuard":"func isValidVarint(buf []byte) bool { _, consumed := protowire.ConsumeVarint(buf); return consumed >= 0 }","tryCatchPattern":"entry, err := ReadOrderedListState(ctx, ...)\nif err != nil && strings.Contains(err.Error(), \"invalid varint\") {\n    return nil, fmt.Errorf(\"state data corrupt or written by incompatible SDK: %w\", err)\n}","preventionTips":["Encode sort keys with protowire.AppendVarint only","Keep writer and reader on compatible Beam versions","Validate state payloads after migration or cache restore"],"tags":["beam-go","state-api","protobuf","varint"],"backgroundTag":"protobuf-unmarshal-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}