nautechsystems/nautilus_trader · error

Failed to finish COPY operation: {e}

Error message

Failed to finish COPY operation: {e}

What it means

After streaming all block rows, the adapter calls `copy_in.finish()` to commit the COPY BINARY stream and finalize the statement. If finishing fails (protocol error, dropped connection mid-stream, server-side constraint failure surfaced at completion), this error is returned. Data may be partially buffered and the COPY is aborted.

Source

Thrown at crates/adapters/blockchain/src/cache/copy.rs:83

            .map_err(|e| anyhow::anyhow!("Failed to start COPY operation: {e}"))?;

        // Write binary header
        self.write_copy_header(&mut copy_in).await?;

        // Write each block as binary data
        for block in blocks {
            self.write_block_binary(&mut copy_in, chain_id, block)
                .await?;
        }

        // Write binary trailer
        self.write_copy_trailer(&mut copy_in).await?;

        // Finish the COPY operation
        copy_in
            .finish()
            .await
            .map_err(|e| anyhow::anyhow!("Failed to finish COPY operation: {e}"))?;

        Ok(())
    }

    /// Inserts tokens using PostgreSQL COPY BINARY for maximum performance.
    ///
    /// # Errors
    ///
    /// Returns an error if the COPY operation fails.
    pub async fn copy_tokens(&self, chain_id: u32, tokens: &[Token]) -> anyhow::Result<()> {
        if tokens.is_empty() {
            return Ok(());
        }

        let copy_statement = "
            COPY token (
                chain_id, address, name, symbol, decimals
            ) FROM STDIN WITH (FORMAT BINARY)";

View on GitHub (pinned to 18893faf8b)

Solutions

1. Check the wrapped `e` for server-reported causes (invalid binary representation, constraint violation) and fix the serialized row data.
2. Verify network stability and Postgres timeouts for long COPY streams; increase statement/idle timeouts if needed.
3. Ensure trailer and row encodings match the declared COPY BINARY column types exactly.
4. Retry the whole `copy_blocks` call; COPY is atomic per statement so partial data is rolled back.

// pre-check column types so binary encodings match:
// SELECT column_name, data_type FROM information_schema.columns WHERE table_name = 'blocks';

if let Err(e) = copy_blocks(&cache, &blocks).await {
    if e.to_string().contains("Failed to finish COPY") {
        log::error!("COPY stream failed at finalize: {e}; retrying batch atomically");
        retry_with_backoff(|| copy_blocks(&cache, &blocks), 3).await?;
    } else { return Err(e); }
}

When it happens

Trigger: `copy_blocks` at crates/adapters/blockchain/src/cache/copy.rs:83 writes header, all block rows, and the trailer, then `copy_in.finish().await` returns Err — typically connection drop during the stream or a server-side data/type error detected on completion.

Common situations: Long-running COPY interrupted by network timeout or Postgres idle/connection limits; binary-encoded values rejected by the server at finalize (bad types, constraint violation); pool connection reclaimed mid-operation.

Related errors

• Failed to start COPY operation: {e}
• Failed to write pool fee collect data: {e}
• Failed to write token data: {e}
• Failed to write pool data: {e}
• Failed to write COPY trailer: {e}

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/0fbee6bd89bfc471. Report an issue: GitHub.