hyperledger/fabric · critical

Could not retrieve header of the last block form file: %s

Error message

Could not retrieve header of the last block form file: %s

What it means

During blockfileMgr startup, after scanning block files, the manager retrieves the header of the last persisted block to seed the blockchain info. If that retrieval fails (corrupt or truncated final block file segment), it panics because the ledger height/hash cannot be determined reliably.

Source

Thrown at common/ledger/blkstorage/blockfile_mgr.go:161

	if err := mgr.syncIndex(); err != nil {
		return nil, err
	}

	bcInfo := &common.BlockchainInfo{}

	if mgr.bootstrappingSnapshotInfo != nil {
		bcInfo.Height = mgr.bootstrappingSnapshotInfo.LastBlockNum + 1
		bcInfo.CurrentBlockHash = mgr.bootstrappingSnapshotInfo.LastBlockHash
		bcInfo.PreviousBlockHash = mgr.bootstrappingSnapshotInfo.PreviousBlockHash
		bcInfo.BootstrappingSnapshotInfo = &common.BootstrappingSnapshotInfo{}
		bcInfo.BootstrappingSnapshotInfo.LastBlockInSnapshot = mgr.bootstrappingSnapshotInfo.LastBlockNum
	}

	if !blockfilesInfo.noBlockFiles {
		lastBlockHeader, err := mgr.retrieveBlockHeaderByNumber(blockfilesInfo.lastPersistedBlock)
		if err != nil {
			panic(fmt.Sprintf("Could not retrieve header of the last block form file: %s", err))
		}
		// update bcInfo with lastPersistedBlock
		bcInfo.Height = blockfilesInfo.lastPersistedBlock + 1
		bcInfo.CurrentBlockHash = protoutil.BlockHeaderHash(lastBlockHeader)
		bcInfo.PreviousBlockHash = lastBlockHeader.PreviousHash
	}
	mgr.bcInfo.Store(bcInfo)
	return mgr, nil
}

func bootstrapFromSnapshottedTxIDs(
	ledgerID string,
	snapshotDir string,
	snapshotInfo *SnapshotInfo,
	conf *Conf,
	indexStore *leveldbhelper.DBHandle,
) error {
	rootDir := conf.getLedgerBlockDir(ledgerID)

View on GitHub (pinned to 2736b63f8f)

Solutions

1. Restore block files from backup taken while the peer was stopped
2. Remove the incomplete trailing data from the newest blockfile segment (truncate to the last complete block boundary) or remove the corrupt segment and re-sync the ledger from ordering service/peer gossip
3. Check file permissions and disk health (fsck) on the ledger data directory
4. As a last resort, reset the ledger and re-fetch blocks from the network

Example fix

// before: panic 'Could not retrieve header of the last block form file'
// after: restore consistent block files
systemctl stop peer
# restore blockfile_000000 files from a known-good backup
rm -rf /var/hyperledger/production/ledgersData/chains/chains/mychannel/index
systemctl start peer  # index rebuilt from consistent files

// verify the newest blockfile ends on a complete block before opening
func lastFileLooksComplete(rootDir string) error {
	files, _ := filepath.Glob(filepath.Join(rootDir, "blockfile_*.log"))
	if len(files) == 0 { return nil }
	info, err := os.Stat(files[len(files)-1])
	if err != nil { return err }
	if info.Size() == 0 { return errors.New("last blockfile is empty/truncated") }
	return nil
}

defer func() {
    if r := recover(); r != nil {
        if strings.Contains(fmt.Sprint(r), "Could not retrieve header of the last block") {
            log.Errorf("last block unreadable/corrupt: %v; restore block files from backup", r)
        }
    }
}()

When it happens

Trigger: newBlockfileMgr with existing non-empty block files where retrieveBlockHeaderByNumber(lastPersistedBlock) fails — typically the last block in the newest blockfile is truncated/incomplete due to a crash mid-write, or the file is unreadable.

Common situations: Peer killed mid-commit leaving a partial block record; disk corruption; block files copied/rsynced while peer was running; manual truncation of blockfile_*.log segments.

Related errors

• error decoding the block number
• error decoding the data hash
• error decoding the previous hash
• error decoding the length of block data
• error decoding the transaction envelope

AI-assisted analysis of hyperledger/fabric@2736b63f8f (2026-09-04). Data as JSON: /api/errors/b62bc21efb7598cb. Report an issue: GitHub.