nats-io/nats-server · error

failed to process trailing gap: %w

Error message

failed to process trailing gap: %w

What it means

After all backup messages are stored, RestoreStreamV2 pads any trailing gap up to nstate.LastSeq with SkipMsgs so deleted trailing messages do not have their sequence numbers reused. This error wraps a failure of that final SkipMsgs call on the destination store.

Source

Thrown at server/stream_backup.go:555

			releaseRestoreBytes(storedSize)
			continue
		}
		if err = store.StoreRawMsg(subj, mhdr, msg, seq, hdr.Timestamp, ttl, false); err != nil {
			return nil, fmt.Errorf("failed to store message sequence %d: %w", seq, err)
		}
		releaseRestoreBytes(storedSize)
	}

	if !eob {
		return mset, fmt.Errorf("backup was truncated")
	}

	// Need to make sure that we pad out with skip msgs to preserve the last
	// sequence, otherwise trailing deleted messages could reuse sequence numbers.
	if lseq < nstate.LastSeq {
		gap := nstate.LastSeq - lseq
		if err := store.SkipMsgs(lseq+1, gap); err != nil {
			return nil, fmt.Errorf("failed to process trailing gap: %w", err)
		}
	}
	return mset, nil
}

View on GitHub (pinned to 3a66a489d2)

Solutions

1. Free disk space / fix storage errors indicated in server logs
2. Delete the partially restored stream and re-run the restore into a fresh stream
3. Verify filesystem health and restart the server so JetStream can reconcile block state
4. Retry the restore once the underlying SkipMsgs/storage error is resolved

if free, err := diskFree(jsStoreDir); err != nil || free < neededBytes { return fmt.Errorf("insufficient JetStream storage") }

if err != nil && strings.Contains(err.Error(), "trailing gap") {
    // storage failure finalizing sequence reservation; fix storage, restore into fresh stream
}

When it happens

Trigger: store.SkipMsgs(lseq+1, gap) fails while padding trailing sequences at the end of restore; driven by storage-layer failures (disk full, I/O error) or an inconsistent destination message store state.

Common situations: Restoring backups of streams with trailing deleted messages onto servers with failing or full storage, or destination streams left inconsistent by a previously failed restore attempt.

Related errors

• message sequence %d out of order
• failed to process gap: %w
• failed to process expired message sequence %d: %w
• failed to store message sequence %d: %w
• storage type can not be updated

AI-assisted analysis of nats-io/nats-server@3a66a489d2 (2026-09-02). Data as JSON: /api/errors/04addfc9394414d9. Report an issue: GitHub.