nautechsystems/nautilus_trader · error · anyhow::Error

Failed to write COPY header: {e}

Error message

Failed to write COPY header: {e}

What it means

write_copy_header sends the 11-byte PostgreSQL COPY BINARY signature, flags, and header-extension-length fields via PgCopyIn::send; failure is wrapped in this error. If the header can't be written the whole COPY stream is unusable and the operation aborts. This is almost always a transport-level failure on the connection, not a data problem.

Source

Thrown at crates/adapters/blockchain/src/cache/copy.rs:328

    /// - 11-byte signature: "PGCOPY\n\xff\r\n\0"
    /// - 4-byte flags field (all zeros)
    /// - 4-byte header extension length (all zeros)
    async fn write_copy_header(
        &self,
        copy_in: &mut sqlx::postgres::PgCopyIn<sqlx::pool::PoolConnection<sqlx::Postgres>>,
    ) -> anyhow::Result<()> {
        use std::io::Write;
        let mut header = Vec::new();

        // PostgreSQL binary copy header
        header.write_all(b"PGCOPY\n\xff\r\n\0")?; // Signature
        header.write_all(&[0, 0, 0, 0])?; // Flags field
        header.write_all(&[0, 0, 0, 0])?; // Header extension length

        copy_in
            .send(header)
            .await
            .map_err(|e| anyhow::anyhow!("Failed to write COPY header: {e}"))?;
        Ok(())
    }

    /// Writes a single block in PostgreSQL binary format.
    ///
    /// Each row in binary format consists of:
    /// - 2-byte field count
    /// - For each field: 4-byte length followed by data (or -1 for NULL)
    async fn write_block_binary(
        &self,
        copy_in: &mut sqlx::postgres::PgCopyIn<sqlx::pool::PoolConnection<sqlx::Postgres>>,
        chain_id: u32,
        block: &Block,
    ) -> anyhow::Result<()> {
        use std::io::Write;
        let mut row_data = Vec::new();

        // Number of fields (14)

View on GitHub (pinned to 18893faf8b)

Solutions

1. Retry the entire copy_* call — the stream is unusable and must be restarted from copy_in_raw
2. Enable PgPoolOptions::test_before_acquire(true) or set a lower max_idle_timeout/max_lifetime
3. Check network path (firewall, LB idle timeouts, pgbouncer server_reset_query)
4. Confirm the DB server did not restart (check server logs) before re-running

Example fix

// before
let pool = PgPoolOptions::new().connect(url).await?;
// after
let pool = PgPoolOptions::new().test_before_acquire(true).idle_timeout(Some(Duration::from_secs(60))).connect(url).await?;

for attempt in 1..=3 {
    match handler.copy_blocks(chain_id, &blocks).await {
        Ok(()) => break,
        Err(e) if attempt < 3 && e.to_string().contains("Failed to write COPY header") => {
            tokio::time::sleep(Duration::from_secs(2u64.pow(attempt))).await;
        }
        Err(e) => return Err(e),
    }
}

When it happens

Trigger: The underlying connection was closed/reset between copy_in_raw and the header send (DB restart, idle timeout, network drop), the server aborted the COPY session, or the driver returned a protocol error.

Common situations: Long-lived PgPool connections killed by a firewall/LB idle timeout; Docker container DB restarted mid-run; pgbouncer recycling connections; running a very long job whose first write happens after an idle gap.

Related errors

• Failed to write block data: {e}
• Failed to write pool swap data: {e}
• Failed to write pool liquidity update data: {e}
• Failed to load active execution intent: {e}
• Failed to start replacement transaction persistence: {e}

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/d4464b00cc6811ea. Report an issue: GitHub.