hyperledger/fabric · error

failed to unmarshal ClusterMetadata: %s

Error message

failed to unmarshal ClusterMetadata: %s

What it means

This error is returned when the chain receives a StepRequest whose metadata payload cannot be deserialized into an etcdraft.ClusterMetadata protobuf. The metadata is sent by the Raft leader to inform followers of active nodes membership; if the bytes are corrupt, truncated, or not valid protobuf, this error is thrown.

Source

Thrown at orderer/consensus/etcdraft/chain.go:531

	}

	if stepMsg.GetTo() != c.raftID {
		c.logger.Warnf("Received msg to %d, my ID is probably wrong due to out of date, cowardly halting", stepMsg.GetTo())
		c.halt()
		return nil
	}

	if err := c.Node.Step(context.TODO(), stepMsg); err != nil {
		return fmt.Errorf("failed to process Raft Step message: %s", err)
	}

	if len(req.GetMetadata()) == 0 || atomic.LoadUint64(&c.lastKnownLeader) != sender { // ignore metadata from non-leader
		return nil
	}

	clusterMetadata := &etcdraft.ClusterMetadata{}
	if err := proto.Unmarshal(req.GetMetadata(), clusterMetadata); err != nil {
		return errors.Errorf("failed to unmarshal ClusterMetadata: %s", err)
	}

	c.Metrics.ActiveNodes.Set(float64(len(clusterMetadata.GetActiveNodes())))
	c.ActiveNodes.Store(clusterMetadata.GetActiveNodes())
	c.logger.Infof("Store ActiveNodes %+v", clusterMetadata.GetActiveNodes())

	return nil
}

// Submit forwards the incoming request to:
// - the local run goroutine if this is leader
// - the actual leader via the transport mechanism
// The call fails if there's no leader elected yet.
func (c *Chain) Submit(req *orderer.SubmitRequest, sender uint64) error {
	if err := c.isRunning(); err != nil {
		c.Metrics.ProposalFailures.Add(1)
		return err
	}

View on GitHub (pinned to 2736b63f8f)

Solutions

1. Verify all ordering nodes run the same Fabric binary version (docker image tags match).
2. Inspect the channel config consensus metadata with 'configtxlator' to confirm it is valid protobuf for etcdraft.Metadata.
3. Capture the inner error from the message to identify whether it is a wire-format or field mismatch.
4. Re-produce the config update: regenerate the channel config block from a clean genesis config and restart the ordering service.

When it happens

Trigger: A leader node broadcasts cluster metadata (active nodes) to followers, but the req.GetMetadata() bytes fail proto.Unmarshal — e.g. version mismatch between ordering nodes, corrupted gRPC payload, or a custom proxy mutating the message.

Common situations: Rolling upgrades where nodes run different Hyperledger Fabric versions with incompatible metadata encoding; a misconfigured intermediary/proxy that mangles protobuf bytes; manually edited consensus metadata in the channel config.

Understand the failure class

Background: "cannot parse invalid wire-format data", "cannot unmarshal", "failed unmarshalling": protobuf unmarshal errors explained — this error's family across 10 libraries.

Related errors

• failed to deserialize values
• error converting envelope to config update: %s
• failed to unmarshal response for transaction %s
• unmarshalling ChaincodeQueryResponse failed
• could not unmarshal chaincode package to CDS or SignedCDS

AI-assisted analysis of hyperledger/fabric@2736b63f8f (2026-09-04). Data as JSON: /api/errors/395f443d883f4f6e. Report an issue: GitHub.