{"record":{"id":"d3c68f783bc5afab","repo":"JuliusBrussee/caveman","slug":"gzip-reader-w","errorCode":null,"errorMessage":"gzip reader: %w","messagePattern":"gzip reader: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"shared/platform/objectstore/compress.go","lineNumber":79,"sourceCode":"\n// DecompressLimit reverses Compress while refusing to produce more than\n// maxBytes of plaintext. The limit is applied while decoding gzip and through\n// zstd's decoder memory bound, so highly-compressible hostile payloads cannot\n// first allocate their full expanded size and only then be rejected.\nfunc DecompressLimit(codec string, data []byte, maxBytes int64) ([]byte, error) {\n\tif maxBytes < 0 || maxBytes > DefaultMaxDecompressedBytes {\n\t\treturn nil, fmt.Errorf(\"invalid decompression limit %d\", maxBytes)\n\t}\n\tswitch codec {\n\tcase CompressionNone, \"\":\n\t\tif int64(len(data)) > maxBytes {\n\t\t\treturn nil, ErrDecompressedTooLarge\n\t\t}\n\t\treturn data, nil\n\tcase CompressionGzip:\n\t\tzr, err := gzip.NewReader(bytes.NewReader(data))\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"gzip reader: %w\", err)\n\t\t}\n\t\tdefer zr.Close()\n\t\tout, err := io.ReadAll(io.LimitReader(zr, maxBytes+1))\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"gzip read: %w\", err)\n\t\t}\n\t\tif int64(len(out)) > maxBytes {\n\t\t\treturn nil, ErrDecompressedTooLarge\n\t\t}\n\t\treturn out, nil\n\tcase CompressionZstd:\n\t\t// Zstd's smallest legal window is 1 KiB. Keep that parser minimum for\n\t\t// tiny valid payloads, then independently enforce the exact byte limit.\n\t\tmemoryLimit := uint64(maxBytes)\n\t\tif memoryLimit < zstd.MinWindowSize {\n\t\t\tmemoryLimit = zstd.MinWindowSize\n\t\t}\n\t\tzr, err := zstd.NewReader(nil,","sourceCodeStart":61,"sourceCodeEnd":97,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/3ee70a102609e550bd2e68004bf5990a9341c851/shared/platform/objectstore/compress.go#L61-L97","documentation":"DecompressLimit could not construct a gzip.Reader from the supplied data, meaning the payload is not a valid gzip stream (bad magic number, truncated header, or corrupt format). The error is wrapped with \"gzip reader: \" and decoding never starts. This is the earliest gzip-specific failure in DecompressLimit and indicates malformed input rather than a size problem.","triggerScenarios":"Calling objectstore.Decompress/DecompressLimit with CompressionGzip on data that was never gzip-compressed (plain bytes, zstd frames, JSON), data truncated during storage/transfer, or a corrupted/stale blob whose stored codec label says gzip but whose bytes are something else.","commonSituations":"Codec metadata drifting from actual stored format after a migration, partial writes or failed uploads persisted, downloading an HTML/JSON error body instead of the compressed object, or hand-edited blobs.","solutions":["Verify the stored codec label matches the actual bytes (gzip streams start with 0x1f 0x8b); re-label or re-compress mismatched objects.","Check the object's integrity: re-download or compare checksums to rule out truncation/corruption during transfer.","Confirm the writer path used Compress(CompressionGzip, ...) and completed without error before the data was stored.","If input may be uncompressed, sniff the magic bytes and call Decompress with CompressionNone when appropriate.","Re-upload the object from source if the stored blob is irrecoverably corrupt."],"exampleFix":"// before\nout, err := objectstore.DecompressLimit(\"gzip\", blob, limit) // gzip reader: gzip: invalid header\n// after\nif len(blob) >= 2 && blob[0] == 0x1f && blob[1] == 0x8b {\n    out, err = objectstore.DecompressLimit(\"gzip\", blob, limit)\n} else {\n    // codec label mismatch or corrupt object: fetch metadata or re-fetch object\n    out, err = objectstore.DecompressLimit(storedCodecFromMetadata, blob, limit)\n}","handlingStrategy":"validation","validationCode":"func isGzip(b []byte) bool {\n    return len(b) >= 2 && b[0] == 0x1f && b[1] == 0x8b\n}\n\nif !isGzip(data) {\n    return fmt.Errorf(\"payload is not gzip (len=%d, magic=%x)\", len(data), data[:min(2, len(data))])\n}\nout, err := objectstore.DecompressLimit(\"gzip\", data, limit)","typeGuard":"func isGzip(b []byte) bool {\n    return len(b) >= 2 && b[0] == 0x1f && b[1] == 0x8b\n}","tryCatchPattern":"if err != nil {\n    if strings.HasPrefix(err.Error(), \"gzip reader:\") {\n        // payload is not a valid gzip stream: check stored codec metadata, re-fetch object\n        return fmt.Errorf(\"corrupt or mislabeled gzip object: %w\", err)\n    }\n    if strings.HasPrefix(err.Error(), \"gzip read:\") {\n        // truncated mid-stream\n        return fmt.Errorf(\"truncated gzip payload: %w\", err)\n    }\n    return err\n}","preventionTips":["Verify checksums on upload/download to catch truncated or corrupted blobs.","Store the codec alongside the object and read it back instead of assuming gzip.","Sniff magic bytes before decompressing data from untrusted or migrated sources.","Ensure writes complete (Compress returned nil error) before persisting blobs."],"tags":["compression","gzip","corruption","go"],"backgroundTag":"invalid-argument-format","analyzedSha":"3ee70a102609e550bd2e68004bf5990a9341c851","analyzedAt":"2026-09-20T15:53:39.229Z","contentChangedAt":"2026-09-20T15:53:39.229Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}