nats-io/nats-server · critical

could not decode avl dmap: %v

Error message

could not decode avl dmap: %v

What it means

This error wraps a failure from avl.Decode() when parsing a delete-map encoded as an AVL seqset (new version message block format). The delete map records which message sequences in the message block have been deleted; if the on-disk bytes are unreadable or malformed, the message block cannot be loaded. NATS JetStream file storage throws this when recovering a stream from disk with a corrupted or truncated delete-map section.

Source

Thrown at server/filestore.go:10191

	}

	// Check for consistency if accounting. If something is off bail and we will rebuild.
	if mb.msgs != (atomic.LoadUint64(&mb.last.seq)-atomic.LoadUint64(&mb.first.seq)+1)-dmapLen {
		os.Remove(ifn)
		return fmt.Errorf("accounting inconsistent")
	}

	// Checksum
	copy(mb.lchk[0:], buf[bi:bi+checksumSize])
	bi += checksumSize

	// Now check for presence of a delete map
	if dmapLen > 0 {
		// New version is encoded avl seqset.
		if buf[1] == newVersion {
			dmap, _, err := avl.Decode(buf[bi:])
			if err != nil {
				return fmt.Errorf("could not decode avl dmap: %v", err)
			}
			mb.dmap = *dmap
		} else {
			// This is the old version.
			for i, fseq := 0, atomic.LoadUint64(&mb.first.seq); i < int(dmapLen); i++ {
				seq := readSeq()
				if seq == 0 {
					break
				}
				mb.dmap.Insert(seq + fseq)
			}
		}
	}

	return nil
}

// Will return total number of cache loads.

View on GitHub (pinned to 3a66a489d2)

Solutions

1. Verify disk integrity (fsck, smartctl) and restore the stream from backup or snapshots.
2. Let JetStream rebuild the stream: delete the stream's data directory (or use stream reset) and republish or re-sync data from upstream.
3. Confirm the filestore version on disk matches the server version; run the same or newer NATS server that wrote the data.
4. Enable JetStream recovery logging and check for preceding I/O errors indicating failing storage hardware.

// Go: check the wrapped cause
if err != nil {
    if strings.Contains(err.Error(), "could not decode avl dmap") {
        log.Printf("stream data corrupt, plan recovery: %v", err)
        // trigger restore / stream rebuild
    }
}

When it happens

Trigger: Loading a message block (mb) during file store recovery where buf[1] == newVersion and dmapLen > 0, and avl.Decode(buf[bi:]) fails due to corrupt/truncated bytes in the delete-map region.

Common situations: Disk corruption or partial writes after a crash; manually copied or truncated stream data directories; version mismatches where a data file was edited or migrated incorrectly.

Related errors

• rebuildState for block %d failed: %w
• bad index file
• metadata incomplete
• unknown compression algorithm
• name required

AI-assisted analysis of nats-io/nats-server@3a66a489d2 (2026-09-02). Data as JSON: /api/errors/74902375528afca4. Report an issue: GitHub.