hyperledger/fabric · error

internal leveldb error while iterating for collection config

Error message

internal leveldb error while iterating for collection config history

What it means

While exporting the collection config history to a snapshot, the namespace LevelDB iterator returned an error mid-iteration. The error is wrapped as an internal leveldb failure since iteration should be reliable absent storage problems.

Source

Thrown at core/ledger/confighistory/mgr.go:234

// block number, and collection config, we store the full key and value to avoid
// unnecessary encoding and decoding of proto messages.
// The key format stored in db is "s" + ns + byte(0) + key + "~collection" + byte(0)
// + blockNum. As we store the key as is, we store 13 extra bytes. For a million
// records, it would add only 12 MB overhead. Note that the protobuf also adds some
// extra bytes. Further, the collection config namespace is not expected to have
// millions of entries.
func (r *Retriever) ExportConfigHistory(dir string, newHashFunc snapshot.NewHashFunc) (map[string][]byte, error) {
	nsItr, err := r.dbHandle.getNamespaceIterator(collectionConfigNamespace)
	if err != nil {
		return nil, err
	}
	defer nsItr.Release()

	var numCollectionConfigs uint64 = 0
	var dataFileWriter *snapshot.FileWriter
	for nsItr.Next() {
		if err := nsItr.Error(); err != nil {
			return nil, errors.Wrap(err, "internal leveldb error while iterating for collection config history")
		}
		if numCollectionConfigs == 0 { // first iteration, create the data file
			dataFileWriter, err = snapshot.CreateFile(filepath.Join(dir, snapshotDataFileName), snapshotFileFormat, newHashFunc)
			if err != nil {
				return nil, err
			}
			defer dataFileWriter.Close()
		}
		if err := dataFileWriter.EncodeBytes(nsItr.Key()); err != nil {
			return nil, err
		}
		if err := dataFileWriter.EncodeBytes(nsItr.Value()); err != nil {
			return nil, err
		}
		numCollectionConfigs++
	}

	if dataFileWriter == nil {

View on GitHub (pinned to 2736b63f8f)

Solutions

1. Check disk space and host I/O health (dmesg, filesystem errors) on the peer node
2. Inspect/repair the confighistory LevelDB (or restore from snapshot) — corruption usually requires rebuilding the ledger
3. Fix directory permissions on the ledger data path for the peer process user
4. Retry snapshot generation after resolving the storage issue

Example fix

# before
df -h /var/hyperledger  # 100% full
# after
free disk space (prune logs/images), then
peer node regenerate snapshot export
# or if DB corrupted: rebuild ledger from snapshot/blockchain

// pre-check host storage before export
if err := checkDiskSpace(dir, minFreeBytes); err != nil {
    return fmt.Errorf("insufficient disk for snapshot export: %w", err)
}

func iteratorHealthy(itr iterator.Iterator) bool {
    return itr.Error() == nil
}

data, err := mgr.ExportConfigHistory(ledgerID, hashFunc, dir)
if err != nil && strings.Contains(err.Error(), "internal leveldb error while iterating") {
    // check disk space / fs errors, then retry export after remediation
    return retryExport(ledgerID, hashFunc, dir)
}

When it happens

Trigger: ExportConfigHistory iterates nsItr and d.GetIterator returns an I/O error during Next() — disk full, corrupted LevelDB files, missing/blocked DB directory, or filesystem permission problems.

Common situations: Disk full or I/O errors on the peer host during snapshot generation; LevelDB corruption after an unclean shutdown; permissions changed on ledgersData directories.

Related errors

• internal leveldb error while iterating for txids
• error while trying to see if the leveldb at path [%s] is emp
• error retrieving leveldb key [%#v]
• error writing leveldb key [%#v]
• error deleting leveldb key [%#v]

AI-assisted analysis of hyperledger/fabric@2736b63f8f (2026-09-04). Data as JSON: /api/errors/53f4cb316322660c. Report an issue: GitHub.