JuliusBrussee/caveman · error

gzip close

Error message

gzip close: %w

What it means

Compress wrote the data into the gzip.Writer successfully but zw.Close() failed. Close flushes remaining buffered data and writes the gzip footer; a failure here means the compressed stream is incomplete and must not be used. The error is wrapped with "gzip close: " and Compress returns nil bytes.

Solutions

1. Retry the compression; a transient close failure on an in-memory buffer should not persist.
2. Treat the returned nil output as authoritative — never fall back to partially-compressed data.
3. Inspect the wrapped cause; if persistent, verify library/binary integrity and memory limits.
4. In tests using a failing writer, confirm the failure is intentional and assert the error path.
5. For large inputs, confirm the process isn't hitting memory limits that could disrupt buffered writes.

Example fix

// before
out, err := objectstore.Compress(objectstore.CompressionGzip, data)
if err != nil { return err } // out is nil, message lost
// after
out, err := objectstore.Compress(objectstore.CompressionGzip, data)
if err != nil {
    return fmt.Errorf("upload aborted: cannot produce compressed payload: %w", err)
}

out, err := objectstore.Compress(objectstore.CompressionGzip, data)
if err != nil {
    if strings.HasPrefix(err.Error(), "gzip close:") {
        // stream incomplete: never use partial output (out is nil anyway); retry
        out, err = objectstore.Compress(objectstore.CompressionGzip, data)
    }
    if err != nil {
        return fmt.Errorf("gzip close failed, payload unusable: %w", err)
    }
}

When it happens

Trigger: Calling objectstore.Compress with CompressionGzip where the underlying writer fails during flush/footer write on Close — again only realistic with a failing underlying writer, corruption, or resource exhaustion since the target is a bytes.Buffer.

Common situations: Rare in production; typically seen with injected failing writers in tests or after unusual memory-pressure conditions aborting the flush.

Related errors

• gzip
• gzip reader
• unknown compression codec
• invalid decompression limit
• objectstore: decompressed payload exceeds limit

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/a5bd78fda21e9c29. Report an issue: GitHub.

Appendix: source

Thrown at shared/platform/objectstore/compress.go:46

// DefaultMaxDecompressedBytes protects callers that do not have trusted size
// metadata. Callers with an expected plaintext size should use
// DecompressLimit with the smaller, metadata-validated limit.
const DefaultMaxDecompressedBytes int64 = 128 << 20

// Compress encodes data with the named codec. "none" returns data unchanged.
func Compress(codec string, data []byte) ([]byte, error) {
	switch codec {
	case CompressionNone, "":
		return data, nil
	case CompressionGzip:
		var buf bytes.Buffer
		zw := gzip.NewWriter(&buf)
		if _, err := zw.Write(data); err != nil {
			return nil, fmt.Errorf("gzip: %w", err)
		}
		if err := zw.Close(); err != nil {
			return nil, fmt.Errorf("gzip close: %w", err)
		}
		return buf.Bytes(), nil
	case CompressionZstd:
		return zstdEnc.EncodeAll(data, nil), nil
	default:
		return nil, fmt.Errorf("unknown compression codec %q", codec)
	}
}

// Decompress reverses Compress with a defensive default expansion ceiling. An
// unknown codec fails closed.
func Decompress(codec string, data []byte) ([]byte, error) {
	return DecompressLimit(codec, data, DefaultMaxDecompressedBytes)
}

// DecompressLimit reverses Compress while refusing to produce more than
// maxBytes of plaintext. The limit is applied while decoding gzip and through
// zstd's decoder memory bound, so highly-compressible hostile payloads cannot

View on GitHub (pinned to 3ee70a1026)