{"record":{"id":"2ab92b4d4c3d4200","repo":"JuliusBrussee/caveman","slug":"objectstore-decompressed-payload-exceeds-limit","errorCode":null,"errorMessage":"objectstore: decompressed payload exceeds limit","messagePattern":"objectstore: decompressed payload exceeds limit","errorType":"exception","errorClass":"ErrDecompressedTooLarge","httpStatus":null,"severity":"error","filePath":"shared/platform/objectstore/compress.go","lineNumber":26,"sourceCode":"\t\"io\"\n\n\t\"github.com/klauspost/compress/zstd\"\n)\n\n// Compression names the supported codecs; they mirror the artifacts.compression\n// CHECK constraint (none|gzip|zstd). An unknown codec fails closed.\nconst (\n\tCompressionNone = \"none\"\n\tCompressionGzip = \"gzip\"\n\tCompressionZstd = \"zstd\"\n)\n\nvar (\n\tzstdEnc, _ = zstd.NewWriter(nil)\n\n\t// ErrDecompressedTooLarge is returned before a compressed payload can\n\t// expand beyond the caller's plaintext ceiling.\n\tErrDecompressedTooLarge = errors.New(\"objectstore: decompressed payload exceeds limit\")\n)\n\n// DefaultMaxDecompressedBytes protects callers that do not have trusted size\n// metadata. Callers with an expected plaintext size should use\n// DecompressLimit with the smaller, metadata-validated limit.\nconst DefaultMaxDecompressedBytes int64 = 128 << 20\n\n// Compress encodes data with the named codec. \"none\" returns data unchanged.\nfunc Compress(codec string, data []byte) ([]byte, error) {\n\tswitch codec {\n\tcase CompressionNone, \"\":\n\t\treturn data, nil\n\tcase CompressionGzip:\n\t\tvar buf bytes.Buffer\n\t\tzw := gzip.NewWriter(&buf)\n\t\tif _, err := zw.Write(data); err != nil {\n\t\t\treturn nil, fmt.Errorf(\"gzip: %w\", err)\n\t\t}","sourceCodeStart":8,"sourceCodeEnd":44,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/3ee70a102609e550bd2e68004bf5990a9341c851/shared/platform/objectstore/compress.go#L8-L44","documentation":"ErrDecompressedTooLarge is a sentinel error returned by DecompressLimit (and the default decompress path for CompressionNone) when a payload would exceed the caller's maxBytes ceiling. It is raised before allocation blowup for compressed data and after a length check for uncompressed data, protecting against decompression-bomb attacks and accidental oversized reads.","triggerScenarios":"Decompressing a zstd payload whose expanded size exceeds maxBytes; passing an uncompressed (CompressionNone/\"\") payload longer than maxBytes; using DefaultMaxDecompressedBytes (128 MiB) implicitly and reading an object larger than that.","commonSituations":"User-supplied compressed uploads (decompression bombs), reading an old object stored before size limits were introduced, decompressing with the default limit instead of a trusted metadata-driven limit.","solutions":["Call DecompressLimit with the trusted plaintext size from your metadata rather than relying on the default","If the data is genuinely needed, raise maxBytes deliberately for that call path","Verify the object's expected size before fetch; reject oversized uploads at ingest time","Check that CompressionNone payloads were stored correctly (an oversized raw payload may indicate a mis-tagged object)"],"exampleFix":"// before\ndata, err := objectstore.DecompressLimit(blob, objectstore.DefaultMaxDecompressedBytes)\n// after\nmaxBytes := int64(obj.Metadata.PlaintextSize) // trusted, validated upstream\ndata, err := objectstore.DecompressLimit(blob, maxBytes)","handlingStrategy":"try-catch","validationCode":"if int64(len(expectedPlaintext)) > maxBytes {\n\treturn errors.New(\"payload exceeds decompression ceiling\")\n}\n// prefer: maxBytes from trusted metadata, not the 128 MiB default","typeGuard":null,"tryCatchPattern":"data, err := objectstore.DecompressLimit(blob, maxBytes)\nif errors.Is(err, objectstore.ErrDecompressedTooLarge) {\n\treturn fmt.Errorf(\"object exceeds %d bytes after decompression\", maxBytes)\n}\nif err != nil {\n\treturn err\n}","preventionTips":["Pass metadata-validated plaintext sizes to DecompressLimit instead of relying on the default","Reject oversized compressed uploads at ingest to prevent decompression bombs","Always check errors with errors.Is against ErrDecompressedTooLarge"],"tags":["objectstore","compression","size-limit"],"backgroundTag":"payload-too-large","analyzedSha":"3ee70a102609e550bd2e68004bf5990a9341c851","analyzedAt":"2026-09-20T15:53:39.229Z","contentChangedAt":"2026-09-20T15:53:39.229Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}