hyperledger/fabric · error · ErrRetryCountExhausted

retry attempts exhausted

Error message

retry attempts exhausted

What it means

ErrRetryCountExhausted is returned when the retry mechanism gave up: pullUntilTarget exceeded its retry attempts without reaching the target block, or GetLatestConfigBlock in util.go exhausted endpoint retries (endpoint == "") or the pulled last block came back nil. It means the cluster node could not be reached or made no progress within the allowed retries.

Source

Thrown at orderer/common/cluster/deliver.go:643

		}, nil
	}
}

type errorAndResponse struct {
	err  error
	resp *orderer.DeliverResponse
}

// ErrForbidden denotes that an ordering node refuses sending blocks due to access control.
var ErrForbidden = errors.New("forbidden pulling the channel")

// ErrServiceUnavailable denotes that an ordering node is not servicing at the moment.
var ErrServiceUnavailable = errors.New("service unavailable")

// ErrNotInChannel denotes that an ordering node is not in the channel
var ErrNotInChannel = errors.New("not in the channel")

var ErrRetryCountExhausted = errors.New("retry attempts exhausted")

View on GitHub (pinned to 2736b63f8f)

Solutions

1. Verify network reachability and TLS trust to all configured cluster endpoints
2. Increase retry-related configuration (retry budget, total time budget) in General.Cluster.RetryTime / puller timeouts
3. Ensure at least one ordering node of the channel is up and serving blocks
4. Check remote node logs for FORBIDDEN/SERVICE_UNAVAILABLE responses that would abort retries early

for _, ep := range endpoints { if conn, err := grpc.Dial(ep, grpc.WithBlock(), grpc.WithTimeout(2*time.Second)); err != nil { log.Printf("endpoint %s unreachable", ep) } }

func isRetryExhausted(err error) bool { return errors.Is(err, cluster.ErrRetryCountExhausted) }

cfgBlock, err := cluster.GetLatestConfigBlock(puller, latestHeight)
if errors.Is(err, cluster.ErrRetryCountExhausted) {
    // all endpoints failed; escalate or back off and retry later
    return err
}

When it happens

Trigger: pullUntilTarget loops past its maximum retry count; util.go:788 — after retrying all endpoints, no endpoint remains (endpoint == ""); util.go:792 — PullBlock returned nil because all pull attempts failed.

Common situations: All ordering endpoints unreachable during eviction detection; persistent network outage or TLS misconfiguration to every cluster member; retry/total timeout configuration too small for a slow cluster; channel stuck because the local node is behind and cannot pull config blocks.

Related errors

• faulty node, received: %v
• response is of type %v, but expected a block
• service unavailable
• not in the channel
• stream %d aborted

AI-assisted analysis of hyperledger/fabric@2736b63f8f (2026-09-04). Data as JSON: /api/errors/8c6ff14466402bdb. Report an issue: GitHub.