nats-io/nats-server · critical

flush of last block failed: %w

Error message

flush of last block failed: %w

What it means

Wrapped by StreamRestore (or Recover/Load) when checkAndFlushLastBlock fails — the in-memory buffered last message block could not be flushed to disk. The underlying cause (I/O error, closed file, encryption error) is wrapped with %w, and the whole thing is passed through storeErr.

Source

Thrown at server/filestore.go:7527

	return nil
}

// This will check all the checksums on messages and report back any sequence numbers with errors.
func (fs *fileStore) checkMsgs() (*LostStreamData, error) {
	fs.mu.Lock()
	defer fs.mu.Unlock()

	var firstErr error
	storeErr := func(err error) error {
		fs.warn("checkMsgs: %v", err)
		if firstErr == nil {
			firstErr = err
		}
		return err
	}

	if err := fs.checkAndFlushLastBlock(); err != nil {
		return nil, storeErr(fmt.Errorf("flush of last block failed: %w", err))
	}

	// Clear any global subject state.
	fs.psim, fs.tsl = fs.psim.Empty(), 0

	for _, mb := range fs.blks {
		// Make sure encryption loaded if needed for the block.
		if err := fs.loadEncryptionForMsgBlock(mb); err != nil {
			_ = storeErr(fmt.Errorf("loading encryption for block %d failed: %w", mb.index, err))
			continue
		}
		// FIXME(dlc) - check tombstones here too?
		ld, _, err := mb.rebuildState()
		if err != nil {
			_ = storeErr(fmt.Errorf("rebuildState for block %d failed: %w", mb.index, err))
			continue
		}
		if ld != nil {

View on GitHub (pinned to 3a66a489d2)

Solutions

1. Inspect the wrapped cause: check free disk space (df -h) and datadir permissions
2. Retry the stream restore after clearing space or fixing mount health
3. Verify no external process holds/locks files in the stream directory
4. Check server logs for the matching storeErr output identifying the failing block

Example fix

// ensure headroom before a large restore
du -sh $NATS_STORE_DIR
# free space / enlarge volume, then re-run:
nats stream restore ORDERS orders.bak

// Go: check headroom before restoring a large stream
if err := checkFreeSpace(storeDir, requiredBytes); err != nil {
    return fmt.Errorf("insufficient space for restore: %w", err)
}

var serr *nats.APIError
if errors.As(err, &serr) && strings.Contains(serr.Description, "flush of last block") {
    if isTransient(serr.Description) { // disk pressure, I/O timeout
        time.Sleep(retryBackoff)
        return restoreStream() // idempotent retry
    }
}

When it happens

Trigger: Restoring a stream from a snapshot, or completing a stream load, where the final block flush to the msg file fails — disk full, I/O error, or the block's file descriptor is invalid.

Common situations: Stream restore finishing on a disk that is full or has become read-only; NFS/EBS timeouts during restore; permissions changed on the stream's datadir mid-operation.

Related errors

• failed to read original block from disk: %w
• failed to create temporary file: %w
• expected state.json contents
• failed to read consumer %q state: %w
• unknown compression algorithm

AI-assisted analysis of nats-io/nats-server@3a66a489d2 (2026-09-02). Data as JSON: /api/errors/8b8631a7866aa8d0. Report an issue: GitHub.