dagger/dagger · error

failed to write compressed diff

Error message

failed to write compressed diff

What it means

While computing the overlay-snapshot diff, the engine walks the upperdir and writes the changed files into the compressed stream (io.MultiWriter of the compressor and a SHA256 digester used for the uncompressed diffID label). If overlay.WriteUpperdir fails, the error is wrapped as 'failed to write compressed diff'. The cause is inside the filesystem diff walk or writing through the compressor.

Source

Thrown at engine/snapshots/blobs_linux.go:79

		}
	}()

	if err = cw.Truncate(0); err != nil {
		return emptyDesc, false, errors.Wrap(err, "failed to truncate writer")
	}

	bufW := bufio.NewWriterSize(cw, 128*1024)
	var labels map[string]string
	if compressorFunc != nil {
		dgstr := digest.SHA256.Digester()
		compressed, err := compressorFunc(bufW, mediaType)
		if err != nil {
			return emptyDesc, false, errors.Wrap(err, "failed to get compressed stream")
		}
		// Close ensure compressorFunc does some finalization works.
		defer compressed.Close()
		if err := overlay.WriteUpperdir(ctx, io.MultiWriter(compressed, dgstr.Hash()), upperdir, lower); err != nil {
			return emptyDesc, false, errors.Wrap(err, "failed to write compressed diff")
		}
		if err := compressed.Close(); err != nil {
			return emptyDesc, false, errors.Wrap(err, "failed to close compressed diff writer")
		}
		labels = map[string]string{
			labelspkg.LabelUncompressed: dgstr.Digest().String(),
		}
	} else {
		if err = overlay.WriteUpperdir(ctx, bufW, upperdir, lower); err != nil {
			return emptyDesc, false, errors.Wrap(err, "failed to write diff")
		}
	}

	if err := bufW.Flush(); err != nil {
		return emptyDesc, false, errors.Wrap(err, "failed to flush diff")
	}
	var commitopts []content.Opt
	if labels != nil {

View on GitHub (pinned to 82ba2681db)

Solutions

1. Read the wrapped inner error for the root cause.
2. Check disk space on the content-store/snapshot volume.
3. Rerun the build — concurrent modification of the snapshot during diff is often transient.
4. If snapshot dirs are corrupt, clear the affected cache and rebuild (e.g. buildctl prune for the ref).

_, ok, err := ref.tryComputeOverlayBlob(ctx, lower, upper, mediaType, refID, compressor)
if err != nil && strings.Contains(err.Error(), "failed to write compressed diff") {
	log.Printf("diff write failed: %v; verifying snapshot and disk", errors.Cause(err))
	// check disk, then retry or fall back to a plain walking differ
}

When it happens

Trigger: overlay.WriteUpperdir(ctx, MultiWriter(compressed, dgstr.Hash()), upperdir, lower) erroring — e.g. upperdir files unreadable/changed mid-walk, whiteout/opaque-xattr handling errors, compressor write failure (disk full) or context cancellation during the diff.

Common situations: Underlying snapshot files removed while the build was running (GC or concurrent prune); disk filling up mid-diff; filesystem corruption in /var/lib/... snapshot dirs; very large layers hitting IO limits.

Related errors

• failed to write diff
• failed to open writer
• failed to truncate writer
• failed to get compressed stream
• failed to close compressed diff writer

AI-assisted analysis of dagger/dagger@82ba2681db (2026-09-05). Data as JSON: /api/errors/5279d6b2c1f9ef83. Report an issue: GitHub.