nats-io/nats-server · critical
loading encryption for block %d failed: %w
Error message
loading encryption for block %d failed: %w
What it means
Emitted during stream recovery (Recover) when loadEncryptionForMsgBlock fails for a given block. The store cannot derive/load the encryption key material for that block's cipher, so the block is skipped (storeErr recorded) and recovery continues with the remaining blocks. Wrapped via %w with the block index.
Source
Thrown at server/filestore.go:7536
storeErr := func(err error) error {
fs.warn("checkMsgs: %v", err)
if firstErr == nil {
firstErr = err
}
return err
}
if err := fs.checkAndFlushLastBlock(); err != nil {
return nil, storeErr(fmt.Errorf("flush of last block failed: %w", err))
}
// Clear any global subject state.
fs.psim, fs.tsl = fs.psim.Empty(), 0
for _, mb := range fs.blks {
// Make sure encryption loaded if needed for the block.
if err := fs.loadEncryptionForMsgBlock(mb); err != nil {
_ = storeErr(fmt.Errorf("loading encryption for block %d failed: %w", mb.index, err))
continue
}
// FIXME(dlc) - check tombstones here too?
ld, _, err := mb.rebuildState()
if err != nil {
_ = storeErr(fmt.Errorf("rebuildState for block %d failed: %w", mb.index, err))
continue
}
if ld != nil {
// Rebuild fs state too.
fs.rebuildStateLocked(ld)
}
if err = fs.populateGlobalPerSubjectInfo(mb); err != nil {
_ = storeErr(fmt.Errorf("populating per-subject info for block %d failed: %w", mb.index, err))
continue
}
}
View on GitHub (pinned to 3a66a489d2)
Solutions
- Check the wrapped cause in server logs to identify the failing block and crypto error
- Verify the stream files were not manually moved between accounts/servers with different encryption keys
- Run integrity checks (nats stream report) and consider removing/re-syncing the corrupt block
- If a single block is lost, delete it (remove .blk/.fst files for that index) so recovery skips only that block
- Restore from backup if the block's data matters
Example fix
// identify and remove a single corrupt block's files, then restart rm /path/jetstream/$ACCOUNT/streams/ORDERS/msgblk/12.blk rm /path/jetstream/$ACCOUNT/streams/ORDERS/msgblk/12.fst
Defensive patterns
Strategy: validation
Validate before calling
// Go: verify stream integrity after recovery
info, err := js.StreamInfo(ctx, "ORDERS")
if err != nil {
return fmt.Errorf("stream recovery issue, check server logs for 'loading encryption for block': %w", err)
} Try / catch
if err != nil {
var serr *nats.APIError
if errors.As(err, &serr) && strings.Contains(serr.Description, "loading encryption for block") {
// block skipped: verify consumer/state consistency before proceeding
return verifyStreamState(js, "ORDERS")
}
return err
} Prevention
- Never manually copy encrypted stream files between accounts or servers
- Use clean shutdowns; avoid kill -9 on storage with encryption enabled
- Run periodic RAID/SMART checks on jetstream volumes
- Keep encrypted stream backups for disaster recovery
When it happens
Trigger: Recovering a stream whose account has encryption enabled but the block's crypto key material is missing/corrupt on disk, or the cipher metadata in the block header is unreadable.
Common situations: Disk corruption or partial writes from an unclean shutdown; restoring encrypted stream files to a different server where key derivation state differs; manually copying stream datadirs across accounts.
Related errors
- rebuildState for block %d failed: %w
- populating per-subject info for block %d failed: %w
- error opening msg block file [%q]: %v
- failed to decompress original block: %w
- unknown compression algorithm
AI-assisted analysis of nats-io/nats-server@3a66a489d2 (2026-09-02).
Data as JSON: /api/errors/77e15e5d62bb5627.
Report an issue: GitHub.