{"record":{"id":"db052e48b888b6cc","repo":"thanos-io/thanos","slug":"short-write-streamed-snappy-header","errorCode":null,"errorMessage":"short-write streamed snappy header","messagePattern":"short-write streamed snappy header","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"pkg/store/postings_codec.go","lineNumber":98,"sourceCode":"\tlength = 5 * length / 4 // estimate 1.25B per posting.\n\n\tblocks := length / maxBlockSize\n\n\tret += blocks * snappy.MaxEncodedLen(maxBlockSize)\n\tlength -= blocks * maxBlockSize\n\tif length > 0 {\n\t\tret += snappy.MaxEncodedLen(length)\n\t}\n\n\treturn ret\n}\n\nfunc diffVarintSnappyStreamedEncode(p index.Postings, length int) ([]byte, error) {\n\tcompressedBuf := bytes.NewBuffer(make([]byte, 0, estimateSnappyStreamSize(length)))\n\tif n, err := compressedBuf.WriteString(codecHeaderStreamedSnappy); err != nil {\n\t\treturn nil, fmt.Errorf(\"writing streamed snappy header\")\n\t} else if n != len(codecHeaderStreamedSnappy) {\n\t\treturn nil, fmt.Errorf(\"short-write streamed snappy header\")\n\t}\n\n\tuvarintEncodeBuf := make([]byte, binary.MaxVarintLen64)\n\n\tsw, err := extsnappy.Compressor.Compress(compressedBuf)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"creating snappy compressor: %w\", err)\n\t}\n\n\tprev := storage.SeriesRef(0)\n\tfor p.Next() {\n\t\tv := p.At()\n\t\tif v < prev {\n\t\t\treturn nil, errors.Errorf(\"postings entries must be in increasing order, current: %d, previous: %d\", v, prev)\n\t\t}\n\n\t\tuvarintSize := binary.PutUvarint(uvarintEncodeBuf, uint64(v-prev))\n\t\tif written, err := sw.Write(uvarintEncodeBuf[:uvarintSize]); err != nil {","sourceCodeStart":80,"sourceCodeEnd":116,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/store/postings_codec.go#L80-L116","documentation":"The streamed snappy encoder verifies the header write reported the full length of codecHeaderStreamedSnappy. A short write means the output buffer did not accept all header bytes, which bytes.Buffer should never do under normal conditions — it signals a broken runtime/invariant.","triggerScenarios":"encodePostingsToCache -> diffVarintSnappyStreamedEncode when WriteString returns n != len(codecHeaderStreamedSnappy); practically only under abnormal buffer/runtime states.","commonSituations":"Buggy custom buffer implementations or exotic memory conditions; virtually never seen in stock Prometheus.","solutions":["Report/investigate the runtime condition; stock bytes.Buffer cannot short-write","Retry cache encoding after freeing resources"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// pre-allocate adequate capacity so bytes.Buffer writes cannot degrade\nbuf := bytes.NewBuffer(make([]byte, 0, estimateSnappyStreamSize(length)+len(codecHeaderStreamedSnappy)))","typeGuard":null,"tryCatchPattern":"b, err := diffVarintSnappyStreamedEncode(p, length)\nif err != nil {\n    if strings.Contains(err.Error(), \"short-write streamed snappy header\") {\n        return nil, fmt.Errorf(\"encoder invariant breached: %w\", err)\n    }\n    return nil, err\n}","preventionTips":["Use stock bytes.Buffer; never substitute custom buffer types in encode paths","Keep the vendored encoding code unmodified","Treat short writes as fatal invariant breaches worth reporting"],"tags":["snappy","encoding","invariant"],"backgroundTag":"internal-invariant-violation","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}