hashicorp/nomad · error

failed to write snapshot file: %v

Error message

failed to write snapshot file: %v

What it means

writeSnapshot wraps errors from write(), which serializes the snapshot metadata and FSM state into the gzip-wrapped archive file. This covers both gzip-compression write failures and the underlying file/disk write errors that occur while producing the snapshot archive. It is raised from New/NewFromFSM before the compressed stream is finalized.

Source

Thrown at helper/snapshot/snapshot.go:124

	defer func() {
		if keep {
			return
		}

		if err := os.Remove(archive.Name()); err != nil {
			logger.Error("Failed to clean up temp snapshot", "error", err)
		}
	}()

	hash := sha256.New()
	out := io.MultiWriter(hash, archive)

	// Wrap the file writer in a gzip compressor.
	compressor := gzip.NewWriter(out)

	// Write the archive.
	if err := write(compressor, metadata, snap); err != nil {
		return nil, fmt.Errorf("failed to write snapshot file: %v", err)
	}

	// Finish the compressed stream.
	if err := compressor.Close(); err != nil {
		return nil, fmt.Errorf("failed to compress snapshot file: %v", err)
	}

	// Sync the compressed file and rewind it so it's ready to be streamed
	// out by the caller.
	if err := archive.Sync(); err != nil {
		return nil, fmt.Errorf("failed to sync snapshot: %v", err)
	}
	if _, err := archive.Seek(0, 0); err != nil {
		return nil, fmt.Errorf("failed to rewind snapshot: %v", err)
	}

	checksum := "sha-256=" + base64.StdEncoding.EncodeToString(hash.Sum(nil))

View on GitHub (pinned to 482b49bf1a)

Solutions

  1. Check free disk space on the temp filesystem and the snapshot storage backend.
  2. Inspect the wrapped cause (%v) — it names the actual failing write; fix that underlying storage/FSM error.
  3. Verify the FSM Persist implementation and its sink are healthy (permissions, backend availability).
  4. Re-run snapshot creation after freeing space or repairing storage; failures here leave the temp file cleaned up automatically.
Defensive patterns

Strategy: try-catch

Validate before calling

func diskHasSpace(min uint64) error {
	var st syscall.Statfs_t
	if err := syscall.Statfs(os.TempDir(), &st); err != nil { return err }
	if uint64(st.Bavail)*uint64(st.Bsize) < min { return fmt.Errorf("low disk on %s", os.TempDir()) }
	return nil
}
// call diskHasSpace(100<<20) before snapshot.New

Try / catch

snap, err := snapshot.New(...)
if err != nil && strings.Contains(err.Error(), "failed to write snapshot file") {
	// err's wrapped cause names the real write failure
	logger.Error("snapshot write failed", "err", err)
	return err
}

Prevention

When it happens

Trigger: New or NewFromFSM where write(compressor, metadata, snap) returns an error: disk full while writing, I/O error on the temp file, the underlying sink's write() (snapshot store or FSM persist) failing, or gzip writer errors from the underlying writer.

Common situations: Disk fills during snapshot creation on the temp volume; storage backend (local store or cloud) returns an I/O error; FSM persist callback errors propagate up wrapped as this message.

Related errors


AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/c897a62b2b45b65f. Report an issue: GitHub.