FuelLabs/fuel-core · error

The block timestamp should monotonically increase

Error message

The block timestamp should monotonically increase

What it means

produce_block enforces self.last_timestamp <= block_time: each new block's Tai64 timestamp must be at least the previous block's. last_timestamp is bootstrapped from the last block header at startup (extract_block_info) and updated after every produced or imported block. Requesting an earlier timestamp is rejected before execution starts.

Source

Thrown at crates/services/consensus_module/poa/src/service.rs:387

        }
        Ok(())
    }

    async fn produce_block(
        &mut self,
        height: BlockHeight,
        block_time: Tai64,
        source: TransactionsSource,
        deadline: Instant,
    ) -> anyhow::Result<()> {
        let last_block_created = Instant::now();
        // verify signing key is set
        if !self.signer.is_available() {
            return Err(anyhow!("unable to produce blocks without a consensus key"))
        }

        if self.last_timestamp > block_time {
            return Err(anyhow!("The block timestamp should monotonically increase"))
        }
        // Ask the block producer to create the block
        let (
            ExecutionResult {
                block,
                skipped_transactions,
                tx_status,
                events,
            },
            changes,
        ) = self
            .signal_produce_block(height, block_time, source, deadline)
            .await?
            .into();

        if !skipped_transactions.is_empty() {
            for (tx_id, err) in skipped_transactions {
                tracing::error!(

View on GitHub (pinned to b9d4d170da)

Solutions

  1. Omit start_time in manually_produce_block so next_time(RequestType::Manual) derives a valid timestamp from last_timestamp plus the elapsed/interval duration.
  2. Sync the node's system clock (NTP/chrony) so clock.now() >= chain head timestamp.
  3. If the head timestamp is in the future, manually produce with start_time above the head timestamp until wall clock catches up.

Example fix

// before: passing a timestamp that can go backwards
let last = last_block_header.time();
node.produce_blocks_with_time(Tai64::now(), 3).await?; // Tai64::now() may be < last after clock drift

// after: let the service derive the next time (or clamp above the head)
node.manually_produce_block(None, Mode::Blocks { number_of_blocks: 3 }).await?;
// or explicitly:
let start = std::cmp::max(Tai64::now(), last);
node.manually_produce_block(Some(start), Mode::Blocks { number_of_blocks: 3 }).await?;
Defensive patterns

Strategy: validation

Validate before calling

// Validate the requested timestamp against the chain head before producing
let last_timestamp = db.latest_header()?.time();
let requested = start_time.unwrap_or_else(|| max Tai64::now(), last_timestamp+1s);
if requested < last_timestamp {
    anyhow::bail!("requested time {requested:?} precedes head timestamp {last_timestamp:?}");
}
node.manually_produce_block(Some(requested), mode).await?;

Type guard

fn timestamp_is_monotonic(last: Tai64, next: Tai64) -> bool {
    next >= last
}

Try / catch

match node.manually_produce_block(Some(t), mode).await {
    Err(err) if err.to_string().contains("monotonically increase") => {
        // re-issue with start_time = None (let the service derive it),
        // or with a timestamp strictly above the chain head
    }
    other => other,
}

Prevention

When it happens

Trigger: manually_produce_block(Some(t), ..) with t earlier than the last block's timestamp; or automatic production where clock.now() (used by next_time(RequestType::Trigger)) is behind the chain's last timestamp — i.e. system clock skew, or a chain whose last block was produced with a future timestamp (e.g. manual blocks with a large start_time).

Common situations: VMs/containers with drifted clocks and no NTP; test chains that jumped timestamps forward via manual start_time then switched back to automatic production; restarting a producer against a chain with future-dated blocks; tests passing non-monotonic start_time values across calls.

Related errors

AI-assisted analysis of FuelLabs/fuel-core@b9d4d170da (2026-08-16). Data as JSON: /api/errors/0429995cf7b2c215. Report an issue: GitHub.