{"record":{"id":"77e15e5d62bb5627","repo":"nats-io/nats-server","slug":"loading-encryption-for-block-d-failed-w","errorCode":null,"errorMessage":"loading encryption for block %d failed: %w","messagePattern":"loading encryption for block (.+?) failed: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"server/filestore.go","lineNumber":7536,"sourceCode":"\tstoreErr := func(err error) error {\n\t\tfs.warn(\"checkMsgs: %v\", err)\n\t\tif firstErr == nil {\n\t\t\tfirstErr = err\n\t\t}\n\t\treturn err\n\t}\n\n\tif err := fs.checkAndFlushLastBlock(); err != nil {\n\t\treturn nil, storeErr(fmt.Errorf(\"flush of last block failed: %w\", err))\n\t}\n\n\t// Clear any global subject state.\n\tfs.psim, fs.tsl = fs.psim.Empty(), 0\n\n\tfor _, mb := range fs.blks {\n\t\t// Make sure encryption loaded if needed for the block.\n\t\tif err := fs.loadEncryptionForMsgBlock(mb); err != nil {\n\t\t\t_ = storeErr(fmt.Errorf(\"loading encryption for block %d failed: %w\", mb.index, err))\n\t\t\tcontinue\n\t\t}\n\t\t// FIXME(dlc) - check tombstones here too?\n\t\tld, _, err := mb.rebuildState()\n\t\tif err != nil {\n\t\t\t_ = storeErr(fmt.Errorf(\"rebuildState for block %d failed: %w\", mb.index, err))\n\t\t\tcontinue\n\t\t}\n\t\tif ld != nil {\n\t\t\t// Rebuild fs state too.\n\t\t\tfs.rebuildStateLocked(ld)\n\t\t}\n\t\tif err = fs.populateGlobalPerSubjectInfo(mb); err != nil {\n\t\t\t_ = storeErr(fmt.Errorf(\"populating per-subject info for block %d failed: %w\", mb.index, err))\n\t\t\tcontinue\n\t\t}\n\t}\n","sourceCodeStart":7518,"sourceCodeEnd":7554,"githubUrl":"https://github.com/nats-io/nats-server/blob/3a66a489d262bf89b71a71c955c94920394532f3/server/filestore.go#L7518-L7554","documentation":"Emitted during stream recovery (Recover) when loadEncryptionForMsgBlock fails for a given block. The store cannot derive/load the encryption key material for that block's cipher, so the block is skipped (storeErr recorded) and recovery continues with the remaining blocks. Wrapped via %w with the block index.","triggerScenarios":"Recovering a stream whose account has encryption enabled but the block's crypto key material is missing/corrupt on disk, or the cipher metadata in the block header is unreadable.","commonSituations":"Disk corruption or partial writes from an unclean shutdown; restoring encrypted stream files to a different server where key derivation state differs; manually copying stream datadirs across accounts.","solutions":["Check the wrapped cause in server logs to identify the failing block and crypto error","Verify the stream files were not manually moved between accounts/servers with different encryption keys","Run integrity checks (nats stream report) and consider removing/re-syncing the corrupt block","If a single block is lost, delete it (remove .blk/.fst files for that index) so recovery skips only that block","Restore from backup if the block's data matters"],"exampleFix":"// identify and remove a single corrupt block's files, then restart\nrm /path/jetstream/$ACCOUNT/streams/ORDERS/msgblk/12.blk\nrm /path/jetstream/$ACCOUNT/streams/ORDERS/msgblk/12.fst","handlingStrategy":"validation","validationCode":"// Go: verify stream integrity after recovery\ninfo, err := js.StreamInfo(ctx, \"ORDERS\")\nif err != nil {\n    return fmt.Errorf(\"stream recovery issue, check server logs for 'loading encryption for block': %w\", err)\n}","typeGuard":null,"tryCatchPattern":"if err != nil {\n    var serr *nats.APIError\n    if errors.As(err, &serr) && strings.Contains(serr.Description, \"loading encryption for block\") {\n        // block skipped: verify consumer/state consistency before proceeding\n        return verifyStreamState(js, \"ORDERS\")\n    }\n    return err\n}","preventionTips":["Never manually copy encrypted stream files between accounts or servers","Use clean shutdowns; avoid kill -9 on storage with encryption enabled","Run periodic RAID/SMART checks on jetstream volumes","Keep encrypted stream backups for disaster recovery"],"tags":["jetstream","filestore","encryption","stream-recovery","data-corruption"],"backgroundTag":"encryption-key-load-failed","analyzedSha":"3a66a489d262bf89b71a71c955c94920394532f3","analyzedAt":"2026-09-02T04:41:54.247Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}