hyperledger/fabric · error

failed to save snapshot to WAL: %s

Error message

failed to save snapshot to WAL: %s

What it means

saveSnap first records a snapshot entry inside the WAL (wal.SaveSnapshot with the snapshot's index/term/confstate) before writing the snapshot file itself. If the WAL write fails, the node cannot durably record where the snapshot was taken, so the snapshot operation is aborted with this wrapped error.

Source

Thrown at orderer/consensus/etcdraft/storage.go:283

	}

	return nil
}

func (rs *RaftStorage) saveSnap(snap *raftpb.Snapshot) error {
	rs.lg.Infof("Persisting snapshot (term: %d, index: %d) to WAL and disk", snap.GetMetadata().Term, snap.GetMetadata().Index)

	// must save the snapshot index to the WAL before saving the
	// snapshot to maintain the invariant that we only Open the
	// wal at previously-saved snapshot indexes.
	walsnap := &walpb.Snapshot{
		Index:     snap.GetMetadata().Index,
		Term:      snap.GetMetadata().Term,
		ConfState: snap.GetMetadata().GetConfState(),
	}

	if err := rs.wal.SaveSnapshot(walsnap); err != nil {
		return errors.Errorf("failed to save snapshot to WAL: %s", err)
	}

	if err := rs.snap.SaveSnap(snap); err != nil {
		return errors.Errorf("failed to save snapshot to disk: %s", err)
	}

	rs.lg.Debugf("Releasing lock to wal files prior to %d", snap.GetMetadata().GetIndex())
	if err := rs.wal.ReleaseLockTo(snap.GetMetadata().GetIndex()); err != nil {
		return err
	}

	return nil
}

// TakeSnapshot takes a snapshot at index i from MemoryStorage, and persists it to wal and disk.
func (rs *RaftStorage) TakeSnapshot(i uint64, cs *raftpb.ConfState, data []byte) error {
	rs.lg.Debugf("Creating snapshot at index %d from MemoryStorage", i)
	snap, err := rs.ram.CreateSnapshot(i, cs, data)

View on GitHub (pinned to 2736b63f8f)

Solutions

1. Check free disk space and filesystem health on the WAL volume
2. Verify the WAL directory exists and is writable by the orderer process
3. Check orderer logs for an earlier error that closed the WAL, and restart the node if storage state is inconsistent
4. Restore from backup / re-provision the node if the storage layer is unrecoverable

Example fix

// before
if err := rs.wal.SaveSnapshot(walsnap); err != nil {
	return errors.Errorf("failed to save snapshot to WAL: %s", err)
}
// after (operator check before restart)
// df -h /var/hyperledger/production/orderer
// chown -R orderer:orderer /var/hyperledger/production/orderer

// operator preflight before starting the orderer
walDir := "/var/hyperledger/production/orderer/channels/<chan>/wal"
if fi, err := os.Stat(walDir); err != nil || !fi.IsDir() {
	log.Fatalf("WAL dir not accessible: %v", err)
}
if err := syscall.Access(walDir, os.O_WRONLY); err != nil {
	log.Fatalf("WAL dir not writable: %v", err)
}
if free, err := freeDiskBytes(walDir); err != nil || free < 1<<30 {
	log.Fatalf("insufficient free disk: %d bytes", free)
}

// wrap TakeSnapshot-triggering path
if err := storage.TakeSnapshot(i, cs, data); err != nil {
	if strings.Contains(err.Error(), "failed to save snapshot to WAL") {
		lg.Errorf("WAL snapshot write failed (disk full/IO?): %s", err)
		// alert ops, do not retry blindly — snapshot must be re-triggered after fix
	}
	return err
}

When it happens

Trigger: rs.wal.SaveSnapshot(walsnap) returns an error — typically disk full, WAL file handle closed, I/O error, or the WAL directory being removed while the node runs during TakeSnapshot.

Common situations: Disk quota exceeded on the orderer's data volume; WAL directory deleted or unmounted; I/O errors from failing hardware; snapshot triggered concurrently with node shutdown closing the WAL.

Related errors

• failed to save snapshot to disk: %s
• failed to repair WAL: %s
• internal leveldb error while iterating for txids
• error while writing data to the snapshot file: %s
• error while flushing to the snapshot file: %s

AI-assisted analysis of hyperledger/fabric@2736b63f8f (2026-09-04). Data as JSON: /api/errors/d65a10925403248e. Report an issue: GitHub.