nats-io/nats-server · critical

could not decode avl dmap

Error message

could not decode avl dmap: %v

What it means

This error wraps a failure from avl.Decode() when parsing a delete-map encoded as an AVL seqset (new version message block format). The delete map records which message sequences in the message block have been deleted; if the on-disk bytes are unreadable or malformed, the message block cannot be loaded. NATS JetStream file storage throws this when recovering a stream from disk with a corrupted or truncated delete-map section.

Solutions

1. Verify disk integrity (fsck, smartctl) and restore the stream from backup or snapshots.
2. Let JetStream rebuild the stream: delete the stream's data directory (or use stream reset) and republish or re-sync data from upstream.
3. Confirm the filestore version on disk matches the server version; run the same or newer NATS server that wrote the data.
4. Enable JetStream recovery logging and check for preceding I/O errors indicating failing storage hardware.

// Go: check the wrapped cause
if err != nil {
    if strings.Contains(err.Error(), "could not decode avl dmap") {
        log.Printf("stream data corrupt, plan recovery: %v", err)
        // trigger restore / stream rebuild
    }
}

When it happens

Trigger: Loading a message block (mb) during file store recovery where buf[1] == newVersion and dmapLen > 0, and avl.Decode(buf[bi:]) fails due to corrupt/truncated bytes in the delete-map region.

Common situations: Disk corruption or partial writes after a crash; manually copied or truncated stream data directories; version mismatches where a data file was edited or migrated incorrectly.

Related errors

• bad index file
• metadata incomplete
• rebuildState for block
• bad ack floor for consumer
• bad ack floor for stream

AI-assisted analysis of nats-io/nats-server@3a66a489d2 (2026-09-02). Data as JSON: /api/errors/74902375528afca4. Report an issue: GitHub.

Appendix: source

Thrown at server/filestore.go:10191

	}

	// Check for consistency if accounting. If something is off bail and we will rebuild.
	if mb.msgs != (atomic.LoadUint64(&mb.last.seq)-atomic.LoadUint64(&mb.first.seq)+1)-dmapLen {
		os.Remove(ifn)
		return fmt.Errorf("accounting inconsistent")
	}

	// Checksum
	copy(mb.lchk[0:], buf[bi:bi+checksumSize])
	bi += checksumSize

	// Now check for presence of a delete map
	if dmapLen > 0 {
		// New version is encoded avl seqset.
		if buf[1] == newVersion {
			dmap, _, err := avl.Decode(buf[bi:])
			if err != nil {
				return fmt.Errorf("could not decode avl dmap: %v", err)
			}
			mb.dmap = *dmap
		} else {
			// This is the old version.
			for i, fseq := 0, atomic.LoadUint64(&mb.first.seq); i < int(dmapLen); i++ {
				seq := readSeq()
				if seq == 0 {
					break
				}
				mb.dmap.Insert(seq + fseq)
			}
		}
	}

	return nil
}

// Will return total number of cache loads.

View on GitHub (pinned to 3a66a489d2)