expected block number is [%d] but actual block number inside

Error message

expected block number is [%d] but actual block number inside block is [%d]

What it means

verifyHeader expects the next block to be exactly lastBlockHeader.Number + 1. A delivered block whose Header.Number does not match the expected sequence breaks the assumption of a contiguous ledger stream, so the assistant rejects it. This protects the client against gaps, replays, and out-of-order delivery.

Source

Thrown at common/deliverclient/block_verification.go:321

func (a *BlockVerificationAssistant) verifyMetadata(block *common.Block) error {
	if block.Metadata == nil || len(block.Metadata.Metadata) < len(common.BlockMetadataIndex_name) {
		return errors.Errorf("block with id [%d] on channel [%s] does not have metadata or contains too few entries", block.Header.Number, a.channelID)
	}

	return nil
}

func (a *BlockVerificationAssistant) verifyHeader(block *common.Block) error {
	if block == nil {
		return errors.Errorf("block must be different from nil, channel=%s", a.channelID)
	}
	if block.Header == nil {
		return errors.Errorf("invalid block, header must be different from nil, channel=%s", a.channelID)
	}

	expectedBlockNum := a.lastBlockHeader.Number + 1
	if expectedBlockNum != block.Header.Number {
		return errors.Errorf("expected block number is [%d] but actual block number inside block is [%d]", expectedBlockNum, block.Header.Number)
	}

	if len(a.lastBlockHeaderHash) != 0 {
		if !bytes.Equal(block.Header.PreviousHash, a.lastBlockHeaderHash) {
			return errors.Errorf("Header.PreviousHash of block [%d] is different from Hash(block.Header) of previous block, on channel [%s], received: %s, expected: %s",
				block.Header.Number, a.channelID, hex.EncodeToString(block.Header.PreviousHash), hex.EncodeToString(a.lastBlockHeaderHash))
		}
	}
	return nil
}

View on GitHub (pinned to 2736b63f8f)

Solutions

Check whether the block is a duplicate; if so, skip it instead of re-verifying against the stale lastBlockHeader.
Reset or re-seed the BlockVerificationAssistant's lastBlockHeader to the correct ledger height after reconnects.
Adjust the SeekInfo start position in the deliver request so the stream begins at the client's actual ledger height.
Compare orderer ledger heights across ordering nodes; resync an orderer that is behind or ahead.

Example fix

// before
env, _ := protoutil.CreateDeliverEnvelope(..., &ab.SeekPosition{Type: &ab.SeekPosition_Newest{}}) // mismatch with local height
// after
startFrom := localLedgerHeight // e.g. fetched from ledger
env, _ := protoutil.CreateDeliverEnvelope(..., &ab.SeekPosition{Type: &ab.SeekPosition_Specified{
    Specified: &ab.SeekSpecified{Start: startFrom},
}})

Defensive patterns

Strategy: retry

Validate before calling

if blk.Header.Number != lastBlockHeader.Number+1 {
    return fmt.Errorf("unexpected block %d; expected %d — re-seek from correct height", blk.Header.Number, lastBlockHeader.Number+1)
}

Try / catch

if err := assistant.VerifyBlock(blk); err != nil {
    var expected uint64
    if n, e := fmt.Sscanf(err.Error(), "expected block number is [%d]", &expected); e == nil && n == 1 {
        // reconnect the deliver stream seeking from `expected`
    }
}

Prevention

Seek deliver streams from your actual local ledger height, not Newest/oldest guesses.
Re-seed lastBlockHeader after every reconnect.
Track duplicates explicitly and skip already-seen block numbers.

When it happens

Trigger: VerifyBlock/VerifyBlockAttestation receives a block whose Header.Number != a.lastBlockHeader.Number + 1 — e.g. a duplicate of the last block, a skipped block, or a block from an earlier point in the ledger arriving on the deliver stream.

Common situations: Orderer restarted/resynced and re-delivered old blocks; deliver client reconnected without resetting lastBlockHeader; Seeking a start block that overlaps already-received blocks; multiple orderers in a cluster out of sync delivering divergent sequences.

Related errors

AI-assisted analysis of hyperledger/fabric@2736b63f8f (2026-09-04). Data as JSON: /api/errors/ffbf7b93416ddf59. Report an issue: GitHub.