{"record":{"id":"53f4cb316322660c","repo":"hyperledger/fabric","slug":"internal-leveldb-error-while-iterating-for-collect","errorCode":null,"errorMessage":"internal leveldb error while iterating for collection config history","messagePattern":"internal leveldb error while iterating for collection config history","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"core/ledger/confighistory/mgr.go","lineNumber":234,"sourceCode":"// block number, and collection config, we store the full key and value to avoid\n// unnecessary encoding and decoding of proto messages.\n// The key format stored in db is \"s\" + ns + byte(0) + key + \"~collection\" + byte(0)\n// + blockNum. As we store the key as is, we store 13 extra bytes. For a million\n// records, it would add only 12 MB overhead. Note that the protobuf also adds some\n// extra bytes. Further, the collection config namespace is not expected to have\n// millions of entries.\nfunc (r *Retriever) ExportConfigHistory(dir string, newHashFunc snapshot.NewHashFunc) (map[string][]byte, error) {\n\tnsItr, err := r.dbHandle.getNamespaceIterator(collectionConfigNamespace)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tdefer nsItr.Release()\n\n\tvar numCollectionConfigs uint64 = 0\n\tvar dataFileWriter *snapshot.FileWriter\n\tfor nsItr.Next() {\n\t\tif err := nsItr.Error(); err != nil {\n\t\t\treturn nil, errors.Wrap(err, \"internal leveldb error while iterating for collection config history\")\n\t\t}\n\t\tif numCollectionConfigs == 0 { // first iteration, create the data file\n\t\t\tdataFileWriter, err = snapshot.CreateFile(filepath.Join(dir, snapshotDataFileName), snapshotFileFormat, newHashFunc)\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t\tdefer dataFileWriter.Close()\n\t\t}\n\t\tif err := dataFileWriter.EncodeBytes(nsItr.Key()); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tif err := dataFileWriter.EncodeBytes(nsItr.Value()); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tnumCollectionConfigs++\n\t}\n\n\tif dataFileWriter == nil {","sourceCodeStart":216,"sourceCodeEnd":252,"githubUrl":"https://github.com/hyperledger/fabric/blob/2736b63f8fd5932511d56fe68b7039d15977f7f6/core/ledger/confighistory/mgr.go#L216-L252","documentation":"While exporting the collection config history to a snapshot, the namespace LevelDB iterator returned an error mid-iteration. The error is wrapped as an internal leveldb failure since iteration should be reliable absent storage problems.","triggerScenarios":"ExportConfigHistory iterates nsItr and d.GetIterator returns an I/O error during Next() — disk full, corrupted LevelDB files, missing/blocked DB directory, or filesystem permission problems.","commonSituations":"Disk full or I/O errors on the peer host during snapshot generation; LevelDB corruption after an unclean shutdown; permissions changed on ledgersData directories.","solutions":["Check disk space and host I/O health (dmesg, filesystem errors) on the peer node","Inspect/repair the confighistory LevelDB (or restore from snapshot) — corruption usually requires rebuilding the ledger","Fix directory permissions on the ledger data path for the peer process user","Retry snapshot generation after resolving the storage issue"],"exampleFix":"# before\ndf -h /var/hyperledger  # 100% full\n# after\nfree disk space (prune logs/images), then\npeer node regenerate snapshot export\n# or if DB corrupted: rebuild ledger from snapshot/blockchain","handlingStrategy":"retry","validationCode":"// pre-check host storage before export\nif err := checkDiskSpace(dir, minFreeBytes); err != nil {\n    return fmt.Errorf(\"insufficient disk for snapshot export: %w\", err)\n}","typeGuard":"func iteratorHealthy(itr iterator.Iterator) bool {\n    return itr.Error() == nil\n}","tryCatchPattern":"data, err := mgr.ExportConfigHistory(ledgerID, hashFunc, dir)\nif err != nil && strings.Contains(err.Error(), \"internal leveldb error while iterating\") {\n    // check disk space / fs errors, then retry export after remediation\n    return retryExport(ledgerID, hashFunc, dir)\n}","preventionTips":["Monitor disk space and I/O health on peer nodes","Ensure clean shutdowns to avoid LevelDB corruption","Fix permissions on ledgersData directories for the peer user","Keep backups/snapshots so a corrupted confighistory DB can be rebuilt"],"tags":["fabric","leveldb","snapshot","storage"],"backgroundTag":"leveldb-iteration-failed","analyzedSha":"2736b63f8fd5932511d56fe68b7039d15977f7f6","analyzedAt":"2026-09-04T08:52:36.465Z","contentChangedAt":"2026-09-04T08:52:36.465Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}