kopia/kopia · error

invalid compression header, expected

Error message

invalid compression header, expected %x but got %x

What it means

The stream's actual 4-byte compression header did not equal the expected header for the compressor performing Decompress. The data was readable, but it was compressed with a different scheme than the one the caller used (or the bytes are not this library's compressed format at all).

Solutions

1. Read the actual 4-byte header and dispatch via compression.DecompressByHeader instead of hard-coding one compressor.
2. Ensure writer and reader use the same configured compression scheme (align config across services/versions).
3. Use the object reader (newRawReader path) which self-selects the compressor, rather than calling a specific compressor directly.
4. Migrate/re-compress legacy blobs to the current scheme.

Example fix

// before
err := compression.ByName["zstd"].Decompress(out, r, true) // header mismatch on snappy blob
// after
err := compression.DecompressByHeader(out, r) // dispatches on actual header

hdr := make([]byte, 4)
if _, err := io.ReadFull(r, hdr); err != nil { return err }
expected := compression.ByName["zstd"].Header() // scheme you intend to use
if !bytes.Equal(hdr, expected) {
    // fall back to header-driven dispatch instead of hard-coded compressor
    r = io.MultiReader(bytes.NewReader(hdr), r)
    return compression.DecompressByHeader(out, r)
}

func matchesScheme(r io.Reader, want []byte) (bool, io.Reader) {
    b := make([]byte, len(want))
    if _, err := io.ReadFull(r, b); err != nil { return false, r }
    return bytes.Equal(b, want), io.MultiReader(bytes.NewReader(b), r)
}

if err := comp.Decompress(out, r, true); err != nil {
    if strings.Contains(err.Error(), "invalid compression header") {
        r.Seek(0, io.SeekStart)
        return compression.DecompressByHeader(out, r) // dispatch by actual header
    }
    return err
}

When it happens

Trigger: Calling a specific compressor's Decompress (withHeader true) on data written by a different compressor — e.g. gzip.Decompress on zstd-compressed bytes; passing plaintext or another format's magic (e.g. gzip's 1f8b) where the library's 4-byte ID header is expected.

Common situations: Config change between write and read (default compression switched from snappy to zstd); reading legacy blobs written with an older scheme; hand-rolled code that strips or misaligns headers before calling Decompress.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

• compression close error
• compression close error
• compression close error
• compression close error
• compression close error

AI-assisted analysis of kopia/kopia@82495e54b5 (2026-09-07). Data as JSON: /api/errors/c3c29e8d46bde2e2. Report an issue: GitHub.

Appendix: source

Thrown at repo/compression/compressor.go:115

		return false
	}

	return !isUnsupported[c.HeaderID()]
}

func mustSucceed(err error) {
	impossible.PanicOnError(err)
}

func verifyCompressionHeader(reader io.Reader, want []byte) error {
	var actual [compressionHeaderSize]byte

	if _, err := io.ReadFull(reader, actual[:]); err != nil {
		return errors.Wrap(err, "error reading compression header")
	}

	if !bytes.Equal(actual[:], want) {
		return errors.Errorf("invalid compression header, expected %x but got %x", want, actual[:])
	}

	return nil
}

View on GitHub (pinned to 82495e54b5)