{"record":{"id":"1433ce5eca0d93bd","repo":"thanos-io/thanos","slug":"unrecognize-postings-format","errorCode":null,"errorMessage":"unrecognize postings format","messagePattern":"unrecognize postings format","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/store/postings_codec.go","lineNumber":47,"sourceCode":"// and Varint is very efficient at encoding small values (values < 128 are encoded as\n// single byte, values < 16384 are encoded as two bytes). Diff + varint reduces postings size\n// significantly (to about 20% of original), snappy then halves it to ~10% of the original.\n\nconst (\n\tcodecHeaderSnappy         = \"dvs\" // As in \"diff+varint+snappy\".\n\tcodecHeaderStreamedSnappy = \"dss\" // As in \"diffvarint+streamed snappy\".\n)\n\nfunc decodePostings(input []byte) (closeablePostings, error) {\n\tvar df func([]byte, bool) (closeablePostings, error)\n\n\tswitch {\n\tcase isDiffVarintSnappyEncodedPostings(input):\n\t\tdf = diffVarintSnappyDecode\n\tcase isDiffVarintSnappyStreamedEncodedPostings(input):\n\t\tdf = diffVarintSnappyStreamedDecode\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"unrecognize postings format\")\n\t}\n\n\treturn df(input, false)\n}\n\n// isDiffVarintSnappyEncodedPostings returns true, if input looks like it has been encoded by diff+varint+snappy codec.\nfunc isDiffVarintSnappyEncodedPostings(input []byte) bool {\n\treturn bytes.HasPrefix(input, []byte(codecHeaderSnappy))\n}\n\n// isDiffVarintSnappyStreamedEncodedPostings returns true, if input looks like it has been encoded by diff+varint+snappy streamed codec.\nfunc isDiffVarintSnappyStreamedEncodedPostings(input []byte) bool {\n\treturn bytes.HasPrefix(input, []byte(codecHeaderStreamedSnappy))\n}\n\n// estimateSnappyStreamSize estimates the number of bytes\n// needed for encoding length postings. Note that in reality\n// the number of bytes needed could be much bigger if postings","sourceCodeStart":29,"sourceCodeEnd":65,"githubUrl":"https://github.com/thanos-io/thanos/blob/35b8b991177def87ed52dcf10f9b6d87f07282c8/pkg/store/postings_codec.go#L29-L65","documentation":"decodePostings inspects the input byte slice for known codec magic headers (diff+varint+snappy, or its streamed variant) and returns this error when none match. It means the cached postings bytes were not produced by any supported encoder — the format is unknown or the data is corrupt.","triggerScenarios":"Calling decodeCachedPostings with bytes lacking a recognized codecHeaderStreamedSnappy or diff-varint-snappy header: data written by an older Prometheus version, corrupt/truncated cache entries, or a manual/foreign encoder writing to the postings cache.","commonSituations":"Downgrading Prometheus so cached postings in the store cache use an unrecognized format; corrupted head/chunk data on disk after an unclean shutdown; manually migrating index-cache contents across versions.","solutions":["Clear the affected postings cache (index-cache) so entries are re-encoded with the current codec","Ensure the Prometheus version reading the cache is >= the version that wrote it","Verify storage integrity; if index data is corrupt, restore from backup or re-ingest","Check for external tools writing directly to the cache files"],"exampleFix":"// before: trusting arbitrary cache bytes\ndf, _ := decodeCachedPostings(b)\n// after: validate header before decoding\nif !isDiffVarintSnappyEncodedPostings(b) && !isDiffVarintSnappyStreamedEncodedPostings(b) {\n    return fmt.Errorf(\"unrecognized postings format, purge cache\")\n}","handlingStrategy":"validation","validationCode":"func decodablePostings(b []byte) bool {\n    return isDiffVarintSnappyEncodedPostings(b) || isDiffVarintSnappyStreamedEncodedPostings(b)\n}","typeGuard":"func hasKnownPostingsHeader(b []byte) bool {\n    return bytes.HasPrefix(b, codecHeaderStreamedSnappy) || isDiffVarintSnappyEncodedPostings(b)\n}","tryCatchPattern":"p, err := decodeCachedPostings(b)\nif err != nil {\n    if strings.Contains(err.Error(), \"unrecognize postings format\") {\n        // evict cache entry, fall back to fetching postings from index\n        return fetchFromIndex(refs)\n    }\n    return err\n}","preventionTips":["Never share index-cache files across Prometheus versions without checking compatibility","Purge caches after version downgrades","Monitor cache integrity; evict entries that fail decoding"],"tags":["postings","codecs","snappy","corruption"],"backgroundTag":"invalid-argument-format","analyzedSha":"35b8b991177def87ed52dcf10f9b6d87f07282c8","analyzedAt":"2026-09-07T01:49:59.689Z","contentChangedAt":"2026-09-07T01:49:59.689Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}