FuelLabs/fuel-core · error · anyhow::Error
write_block timed out after {:?}
Error message
write_block timed out after {:?} What it means
A single-node Redis write in publish_block_on_node exceeded its configured write_block timeout. The elapsed deadline (included via {:?}) triggers before the Redis client returns, so the outcome of the actual write on that node is unknown — it may have committed after the deadline. Distinct from the FENCING_ERROR branch handled above it; this is a plain timeout error propagating up after the cached connection for the node is cleared.
Source
Thrown at crates/fuel-core/src/service/adapters/consensus_module/poa.rs:1352
}
Ok(Err(err)) if err.to_string().contains("FENCING_ERROR:") => {
poa_metrics().write_block_fencing_error_total.inc();
tracing::warn!(
"write_block: fencing rejected (height={block_height}): {err}"
);
Ok(WriteBlockResult::FencingRejected)
}
Ok(Err(err)) => {
poa_metrics().write_block_error_total.inc();
RedisLeaderLeaseAdapter::clear_cached_connection_for(&self.redis_node)
.await;
Err(err.into())
}
Err(_) => {
poa_metrics().write_block_error_total.inc();
RedisLeaderLeaseAdapter::clear_cached_connection_for(&self.redis_node)
.await;
Err(anyhow!(
"write_block timed out after {:?}",
self.node_timeout
))
}
}
}
async fn check_lease_owner(&self) -> bool {
let mut connection = match RedisLeaderLeaseAdapter::multiplexed_connection_for(
&self.redis_node,
self.node_timeout,
)
.await
{
Ok(connection) => connection,
Err(_) => return false,
};
let is_owner = timeout(View on GitHub (pinned to add100d30d)
Solutions
- Retry the write on the same node — idempotent block writes make a second attempt safe even if the first eventually committed
- Clear/re-establish the Redis connection for the node (the code already clears the cached connection) to drop a possibly stale socket
- Tune the write_block timeout to match observed Redis latency (write_block_error_total and fencing metrics help)
- Rely on the quorum logic: one timed-out node only matters if it drops writes below quorum, in which case repair_sub_quorum_block re-publishes
Defensive patterns
Strategy: retry
When it happens
Trigger: Thrown at crates/fuel-core/src/service/adapters/consensus_module/poa.rs:1352 when the library encounters an invalid state.
Common situations: See trigger scenarios.
Understand the failure class
- Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.
AI-assisted analysis of FuelLabs/fuel-core@add100d30d (2026-09-05).
Data as JSON: /api/errors/4aad633dc1cacaf0.
Report an issue: GitHub.