nats-io/nats-server · error

populating per-subject info for block %d failed: %w

Error message

populating per-subject info for block %d failed: %w

What it means

Emitted during stream recovery when populateGlobalPerSubjectInfo fails for a block. After rebuildState succeeds, the store builds per-subject index info for subject-filtered consumers; failing here means the block's subject metadata could not be processed. The error is recorded via storeErr and that block is skipped in the per-subject pass.

Source

Thrown at server/filestore.go:7550

	for _, mb := range fs.blks {
		// Make sure encryption loaded if needed for the block.
		if err := fs.loadEncryptionForMsgBlock(mb); err != nil {
			_ = storeErr(fmt.Errorf("loading encryption for block %d failed: %w", mb.index, err))
			continue
		}
		// FIXME(dlc) - check tombstones here too?
		ld, _, err := mb.rebuildState()
		if err != nil {
			_ = storeErr(fmt.Errorf("rebuildState for block %d failed: %w", mb.index, err))
			continue
		}
		if ld != nil {
			// Rebuild fs state too.
			fs.rebuildStateLocked(ld)
		}
		if err = fs.populateGlobalPerSubjectInfo(mb); err != nil {
			_ = storeErr(fmt.Errorf("populating per-subject info for block %d failed: %w", mb.index, err))
			continue
		}
	}

	return fs.ld, firstErr
}

// Lock should be held.
func (mb *msgBlock) enableForWriting(fip bool) error {
	if mb == nil {
		return errNoMsgBlk
	}
	if mb.mfd != nil {
		return nil
	}
	mb.fs.dios.acquire()
	mfd, err := os.OpenFile(mb.mfn, os.O_CREATE|os.O_RDWR, defaultFilePerms)
	mb.fs.dios.release()

View on GitHub (pinned to 3a66a489d2)

Solutions

1. Check the wrapped cause in server logs for the failing block and underlying error
2. Restart the server to retry recovery after any transient storage issue
3. If one block is consistently bad, remove that block's files and re-create/resync the stream
4. Verify no NATS version downgrade occurred (older binary reading newer on-disk format); align versions
5. Restore the stream from backup if per-subject data is required

Example fix

// align server version before restarting recovery
nats-server -v
# upgrade/downgrade to the version that wrote the stream, then restart
systemctl restart nats-server

// Go: after recovery, verify per-subject counts for filtered consumers
info, err := js.StreamInfo(ctx, "ORDERS")
if err == nil {
    for subj, n := range info.State.Subjects {
        if n == 0 && expectMsgs(subj) {
            log.Printf("subject %s lost messages — per-subject info skipped for a block", subj)
        }
    }
}

if err != nil {
    if strings.Contains(err.Error(), "populating per-subject info for block") {
        // filtered consumers may miss msgs in the skipped block
        return repopulateOrResync(js, "ORDERS")
    }
    return err
}

When it happens

Trigger: Recover() on a stream with multi-subject (wildcard/multi-filter) data where the block's subject index entries are corrupt or inconsistent with the rebuilt state.

Common situations: Corruption from unclean shutdown on a stream that used many distinct subjects; OOM/interrupted writes while subject maps were being flushed; version mismatches in on-disk subject index format after an upgrade.

Related errors

• loading encryption for block %d failed: %w
• rebuildState for block %d failed: %w
• error opening msg block file [%q]: %v
• unknown compression algorithm
• name required

AI-assisted analysis of nats-io/nats-server@3a66a489d2 (2026-09-02). Data as JSON: /api/errors/29c3206ac393974b. Report an issue: GitHub.