hyperledger/fabric · critical

error getting ledger ids from idStore

Error message

error getting ledger ids from idStore

What it means

After iterating ledger metadata records, the idStore checks the iterator's accumulated error via itr.Error(). If the underlying LevelDB iterator hit an I/O or corruption error mid-scan, the ledger ID enumeration fails with this wrapped error. Unlike error 1030 this is about the iterator itself failing, not a specific malformed record.

Source

Thrown at core/ledger/kvledger/kv_ledger_provider.go:718

func (s *idStore) getLedgerIDs(filterIn map[msgs.Status]struct{}) ([]string, error) {
	var ids []string
	itr := s.db.GetIterator(metadataKeyPrefix, metadataKeyStop)
	defer itr.Release()
	for itr.Error() == nil && itr.Next() {
		metadata := &msgs.LedgerMetadata{}
		if err := proto.Unmarshal(itr.Value(), metadata); err != nil {
			logger.Errorf("Error unmarshalling ledger metadata: %s", err)
			return nil, errors.Wrapf(err, "error unmarshalling ledger metadata")
		}
		if _, ok := filterIn[metadata.Status]; ok {
			id := ledgerIDFromMetadataKey(itr.Key())
			ids = append(ids, id)
		}
	}
	if err := itr.Error(); err != nil {
		logger.Errorf("Error getting ledger ids from idStore: %s", err)
		return nil, errors.Wrapf(err, "error getting ledger ids from idStore")
	}
	return ids, nil
}

func (s *idStore) close() {
	s.db.Close()
}

func ledgerIDFromGenesisBlockKey(key []byte) string {
	return string(key[len(genesisBlkKeyPrefix):])
}

func metadataKey(ledgerID string) []byte {
	return append(metadataKeyPrefix, []byte(ledgerID)...)
}

func ledgerIDFromMetadataKey(key []byte) string {
	return string(key[len(metadataKeyPrefix):])

View on GitHub (pinned to 2736b63f8f)

Solutions

1. Check disk space, permissions, and filesystem health on the peer's ledger data directory (Peer.FileSystemPath/ledgersData).
2. Fix the ownership/permissions so the user running the peer can read/write ledgersData/ledgerProvider.
3. Restore the ledgerProvider DB from backup if files are corrupted.
4. Run 'peer node rebuild-dbs' to rebuild indexes/state from the block store if the DB cannot be repaired.
5. Restart the peer and retry once the environment issue is resolved.

// verify ledger data dir health pre-start
func checkLedgerDataDir(rootFSPath string) error {
	if err := os.MkdirAll(rootFSPath, 0o750); err != nil {
		return fmt.Errorf("cannot create/access ledger dir: %w", err)
	}
	test := filepath.Join(rootFSPath, ".writetest")
	if err := os.WriteFile(test, []byte("ok"), 0o600); err != nil {
		return fmt.Errorf("ledger dir not writable: %w", err)
	}
	return os.Remove(test)
}

ids, err := provider.ListLedgers()
if err != nil && strings.Contains(err.Error(), "error getting ledger ids from idStore") {
	if isIOError(err) {
		// alert ops: disk/permission problem on ledger volume
		monitoring.Alert("ledger-idstore-io", err)
	}
	return err
}

When it happens

Trigger: Any call to getActiveLedgerIDs or getActiveAndInactiveLedgerIDs (peer startup, snapshot-related checks, upgrade) where the LevelDB iterator over the metadata prefix raises an error during traversal.

Common situations: Disk full or I/O errors on the ledger data volume; corrupted MANIFEST/.ldb files in ledgersData/ledgerProvider; permission problems on the DB directory after running commands as different users (e.g. root vs. the peer service user).

Related errors

• internal leveldb error while retrieving data from db iterato
• error in block index: %s
• no such block hash [%x] in index
• error while trying to check the presence of TXID [%s]
• error while unmarshalling signable metadata

AI-assisted analysis of hyperledger/fabric@2736b63f8f (2026-09-04). Data as JSON: /api/errors/88fb9dbaa26d27e6. Report an issue: GitHub.