What causes "Failed to write pool fee collect data: {e}" in nautechsystems/nautilus_trader?

Inspect the chained {e} source for the concrete sqlx/postgres error (connection vs encoding)

nautechsystems/nautilus_trader · error · anyhow::Error

Failed to write pool fee collect data: {e}

Error message

Failed to write pool fee collect data: {e}

What it means

write_pool_fee_collect_binary serializes one pool fee collect row in PostgreSQL COPY binary format and sends it through the copy_in sink. This error wraps the sink's send error when the row bytes cannot be delivered to the server. It is per-row; the underlying cause is usually a connection problem or a server-side rejection of the COPY stream.

Source

Thrown at crates/adapters/blockchain/src/cache/copy.rs:762

        let owner_bytes = collect.owner.to_string().as_bytes().to_vec();
        row_data.write_all(&(owner_bytes.len() as i32).to_be_bytes())?;
        row_data.write_all(&owner_bytes)?;

        write_copy_numeric(&mut row_data, collect.amount0);
        write_copy_numeric(&mut row_data, collect.amount1);

        let tick_lower_bytes = collect.tick_lower.to_be_bytes();
        row_data.write_all(&(tick_lower_bytes.len() as i32).to_be_bytes())?;
        row_data.write_all(&tick_lower_bytes)?;

        let tick_upper_bytes = collect.tick_upper.to_be_bytes();
        row_data.write_all(&(tick_upper_bytes.len() as i32).to_be_bytes())?;
        row_data.write_all(&tick_upper_bytes)?;

        copy_in
            .send(row_data)
            .await
            .map_err(|e| anyhow::anyhow!("Failed to write pool fee collect data: {e}"))?;
        Ok(())
    }

    /// Writes a single token in PostgreSQL binary format.
    ///
    /// Each row in binary format consists of:
    /// - 2-byte field count
    /// - For each field: 4-byte length followed by data (or -1 for NULL)
    async fn write_token_binary(
        &self,
        copy_in: &mut sqlx::postgres::PgCopyIn<sqlx::pool::PoolConnection<sqlx::Postgres>>,
        chain_id: u32,
        token: &Token,
    ) -> anyhow::Result<()> {
        use std::io::Write;
        let mut row_data = Vec::new();

        row_data.write_all(&5u16.to_be_bytes())?;

View on GitHub (pinned to 18893faf8b)

Solutions

Inspect the chained {e} source for the concrete sqlx/postgres error (connection vs encoding)
Confirm the table schema still matches the binary field order and types used by write_pool_fee_collect_binary
Check network stability and increase TCP keepalive / remove short timeouts for bulk loads
Retry the whole copy_pool_collects operation after the connection recovers

Example fix

// before: unbounded loop without backpressure handling can stall/fail on dropped connection
for c in collects { write_pool_fee_collect_binary(&mut copy_in, c).await?; }
// after: chunk and retry on transient errors
for chunk in collects.chunks(10_000) {
    for c in chunk {
        write_pool_fee_collect_binary(&mut copy_in, c).await
            .map_err(|e| e.context("pool fee collect row"))?;
    }
}

Defensive patterns

Strategy: retry

Validate before calling

// pre-validate row against expected schema before streaming
fn validate_collect(c: &PoolFeeCollect) -> Result<(), String> {
    if c.amount.is_negative() { return Err("negative collect amount".into()); }
    Ok(())
}

Try / catch

for chunk in collects.chunks(5_000) {
    if let Err(e) = copy_collect_chunk(db, chain, chunk).await {
        if is_connection_error(&e) {
            tokio::time::sleep(Duration::from_secs(2)).await;
            copy_collect_chunk(db, chain, chunk).await?;
        } else { return Err(e); }
    }
}

Prevention

Validate collect values (ticks, amounts) before serializing to binary format
Keep schema migrations in lockstep with the adapter's binary row encoding
Use TCP keepalive on the Postgres connection for long-running syncs
Chunk large row sets so failures are easier to isolate and retry

When it happens

Trigger: copy_in.send(row_data) fails inside write_pool_fee_collect_binary while copy_pool_collects streams rows — connection closed, server aborted the COPY, or a prior row caused the channel to error.

Common situations: Database restart or failover during backfill; statement/network timeout on a long COPY; schema drift making the binary encoding of tick_upper or amount fields invalid for the target column type.

Related errors

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/0aa832018d3597ef. Report an issue: GitHub.