nats-io/nats-server · error

error writing to compression writer: %w

Error message

error writing to compression writer: %w

What it means

During Compress(), io.CopyN streams the block body into the s2 writer; if the copy reports an error, the underlying write failed and the block cannot be produced, so the error is wrapped with "error writing to compression writer: %w". From server/filestore.go.

Source

Thrown at server/filestore.go:14521

	bodyLen := int64(len(buf) - checksumSize)
	var output bytes.Buffer
	var writer io.WriteCloser
	switch alg {
	case NoCompression:
		return buf, nil
	case S2Compression:
		writer = s2.NewWriter(&output)
	default:
		return nil, fmt.Errorf("compression algorithm not known")
	}

	input := bytes.NewReader(buf[:bodyLen])
	checksum := buf[bodyLen:]

	// Compress the block content, but don't compress the checksum.
	// We will preserve it at the end of the block as-is.
	if n, err := io.CopyN(writer, input, bodyLen); err != nil {
		return nil, fmt.Errorf("error writing to compression writer: %w", err)
	} else if n != bodyLen {
		return nil, fmt.Errorf("short write on body (%d != %d)", n, bodyLen)
	}
	if err := writer.Close(); err != nil {
		return nil, fmt.Errorf("error closing compression writer: %w", err)
	}

	// Now add the checksum back onto the end of the block.
	if n, err := output.Write(checksum); err != nil {
		return nil, fmt.Errorf("error writing checksum: %w", err)
	} else if n != checksumSize {
		return nil, fmt.Errorf("short write on checksum (%d != %d)", n, checksumSize)
	}

	return output.Bytes(), nil
}

func (alg StoreCompression) Decompress(buf []byte) ([]byte, error) {

View on GitHub (pinned to 3a66a489d2)

Solutions

1. Inspect the wrapped %w cause to find the underlying writer failure
2. Retry the compression once with a fresh output buffer (transient memory pressure)
3. Reduce block size to lower peak compression memory usage
4. If persistent, check host memory limits and disable compression (NoCompression) as a fallback

null

out, err := alg.Compress(block)
if err != nil {
	var werr error
	if strings.Contains(err.Error(), "error writing to compression writer") {
		// unwrap: retry once with fresh buffer, then fall back to NoCompression
		werr = retryCompressOrNoop(block)
	}
	_ = werr
	return err
}

When it happens

Trigger: io.CopyN returns err != nil while compressing a block body — e.g. the s2 writer's internal buffer flush fails on the underlying bytes.Buffer (allocation failure) or the input reader fails early.

Common situations: Out-of-memory / allocation pressure making the output buffer fail; programming error passing a nil or closed writer; extremely large blocks stressing memory limits.

Related errors

• short write on body (%d != %d)
• failed to read original block from disk: %w
• failed to read existing metadata header: %w
• failed to decompress original block: %w
• failed to compress block: %w

AI-assisted analysis of nats-io/nats-server@3a66a489d2 (2026-09-02). Data as JSON: /api/errors/f015449860bbfdc4. Report an issue: GitHub.