hyperledger/fabric · error

failed to save snapshot to disk: %s

Error message

failed to save snapshot to disk: %s

What it means

saveSnap writes the snapshot file to the snapshot directory via rs.snap.SaveSnap after the WAL entry succeeds. If persisting the snapshot file to disk fails, this error is returned, leaving the WAL snapshot entry present but the snapshot data missing on disk.

Source

Thrown at orderer/consensus/etcdraft/storage.go:287

func (rs *RaftStorage) saveSnap(snap *raftpb.Snapshot) error {
	rs.lg.Infof("Persisting snapshot (term: %d, index: %d) to WAL and disk", snap.GetMetadata().Term, snap.GetMetadata().Index)

	// must save the snapshot index to the WAL before saving the
	// snapshot to maintain the invariant that we only Open the
	// wal at previously-saved snapshot indexes.
	walsnap := &walpb.Snapshot{
		Index:     snap.GetMetadata().Index,
		Term:      snap.GetMetadata().Term,
		ConfState: snap.GetMetadata().GetConfState(),
	}

	if err := rs.wal.SaveSnapshot(walsnap); err != nil {
		return errors.Errorf("failed to save snapshot to WAL: %s", err)
	}

	if err := rs.snap.SaveSnap(snap); err != nil {
		return errors.Errorf("failed to save snapshot to disk: %s", err)
	}

	rs.lg.Debugf("Releasing lock to wal files prior to %d", snap.GetMetadata().GetIndex())
	if err := rs.wal.ReleaseLockTo(snap.GetMetadata().GetIndex()); err != nil {
		return err
	}

	return nil
}

// TakeSnapshot takes a snapshot at index i from MemoryStorage, and persists it to wal and disk.
func (rs *RaftStorage) TakeSnapshot(i uint64, cs *raftpb.ConfState, data []byte) error {
	rs.lg.Debugf("Creating snapshot at index %d from MemoryStorage", i)
	snap, err := rs.ram.CreateSnapshot(i, cs, data)
	if err != nil {
		return errors.Errorf("failed to create snapshot from MemoryStorage: %s", err)
	}

View on GitHub (pinned to 2736b63f8f)

Solutions

1. Check disk space on the snapshot data volume and free space if needed
2. Verify the snapshots directory exists and is writable by the orderer process
3. Inspect orderer logs for the underlying I/O error and fix storage hardware/mount issues
4. Restart the node; if state is inconsistent, restore from backup or re-join the consenter

Example fix

// before
if err := rs.snap.SaveSnap(snap); err != nil {
	return errors.Errorf("failed to save snapshot to disk: %s", err)
}
// after (operator check)
// df -h /var/hyperledger/production/orderer/snapshots
// mkdir -p <snapshots-dir> && chown orderer:orderer <snapshots-dir>

// operator preflight on the snapshot directory
snapDir := "/var/hyperledger/production/orderer/channels/<chan>/snapshots"
if err := os.MkdirAll(snapDir, 0o755); err != nil {
	log.Fatalf("cannot create snapshot dir: %v", err)
}
if err := syscall.Access(snapDir, os.O_WRONLY); err != nil {
	log.Fatalf("snapshot dir not writable by orderer user: %v", err)
}

if err := storage.TakeSnapshot(i, cs, data); err != nil {
	if strings.Contains(err.Error(), "failed to save snapshot to disk") {
		lg.Errorf("snapshot file write failed: %s", err)
		// check ENOSPC / EACCES on snapshots dir before restarting
	}
	return err
}

When it happens

Trigger: rs.snap.SaveSnap(snap) fails — snapshot directory missing/unwritable, disk full, or an I/O error (bad sector, ENOSPC, permission denied) while writing the snapshot file.

Common situations: Volume filled between WAL write and snapshot write; snapshot directory removed by cleanup scripts or wrong file permissions after a container image change; failing disk.

Related errors

• failed to save snapshot to WAL: %s
• error while writing data to the snapshot file: %s
• internal leveldb error while iterating for txids
• error while flushing to the snapshot file: %s
• error while closing the snapshot file: %s

AI-assisted analysis of hyperledger/fabric@2736b63f8f (2026-09-04). Data as JSON: /api/errors/6a20c8a31099c994. Report an issue: GitHub.