What causes "gzip: %w" in JuliusBrussee/caveman?

Retry the compression call — a one-off write failure on an in-memory buffer is not reproducible.

gzip

Error message

gzip: %w

What it means

Compress failed while writing input data into a gzip.Writer backed by an in-memory buffer. gzip's Write only errors if the underlying writer fails or internal state is corrupted, which for a bytes.Buffer is essentially impossible in normal use; seeing this indicates a corrupted compressor or an out-of-memory/panic-like condition surfaced as an error. It is wrapped with the "gzip: " prefix and returned with a nil result.

Solutions

Retry the compression call — a one-off write failure on an in-memory buffer is not reproducible.
Inspect the wrapped error to identify the real cause; if it persists, check for corrupted binary/dependency versions.
Ensure you are passing a valid []byte to Compress (nil is fine; it means empty input).
If this recurs in tests, check whether a custom io.Writer is being injected that can fail.
Keep the compressed-data path simple: pass data directly to Compress rather than wrapping writers yourself.

Example fix

// before
out, err := objectstore.Compress(objectstore.CompressionGzip, data) // gzip: write failed
// after
out, err := objectstore.Compress(objectstore.CompressionGzip, data)
if err != nil {
    return fmt.Errorf("compressing object before upload: %w", err) // add context for diagnosis
}

Defensive patterns

Strategy: try-catch

Try / catch

out, err := objectstore.Compress(codec, data)
if err != nil {
    // gzip write failures on an in-memory buffer are effectively non-reproducible:
    // log and retry once, then surface the wrapped cause
    out, err = objectstore.Compress(codec, data)
    if err != nil {
        return fmt.Errorf("compression failed: %w", err)
    }
}

Prevention

Pass data directly to Compress; do not inject custom failing writers.
Pin library/dependency versions to avoid corrupted builds.
Retry once on transient write failures before surfacing the error.
Monitor process memory if compressing very large inputs.

When it happens

Trigger: Calling objectstore.Compress with codec CompressionGzip where zw.Write(data) returns a non-nil error — realistically only from a corrupted gzip.Writer state, exotic Writer wrappers, or resource exhaustion.

Common situations: Very rare in practice because bytes.Buffer never fails; may appear in tests with injected failure writers, or after refactors that swap the buffer for a failing io.Writer.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/a711152751e4100d. Report an issue: GitHub.

Appendix: source

Thrown at shared/platform/objectstore/compress.go:43

	// expand beyond the caller's plaintext ceiling.
	ErrDecompressedTooLarge = errors.New("objectstore: decompressed payload exceeds limit")
)

// DefaultMaxDecompressedBytes protects callers that do not have trusted size
// metadata. Callers with an expected plaintext size should use
// DecompressLimit with the smaller, metadata-validated limit.
const DefaultMaxDecompressedBytes int64 = 128 << 20

// Compress encodes data with the named codec. "none" returns data unchanged.
func Compress(codec string, data []byte) ([]byte, error) {
	switch codec {
	case CompressionNone, "":
		return data, nil
	case CompressionGzip:
		var buf bytes.Buffer
		zw := gzip.NewWriter(&buf)
		if _, err := zw.Write(data); err != nil {
			return nil, fmt.Errorf("gzip: %w", err)
		}
		if err := zw.Close(); err != nil {
			return nil, fmt.Errorf("gzip close: %w", err)
		}
		return buf.Bytes(), nil
	case CompressionZstd:
		return zstdEnc.EncodeAll(data, nil), nil
	default:
		return nil, fmt.Errorf("unknown compression codec %q", codec)
	}
}

// Decompress reverses Compress with a defensive default expansion ceiling. An
// unknown codec fails closed.
func Decompress(codec string, data []byte) ([]byte, error) {
	return DecompressLimit(codec, data, DefaultMaxDecompressedBytes)
}

View on GitHub (pinned to 3ee70a1026)