{"record":{"id":"d4464b00cc6811ea","repo":"nautechsystems/nautilus_trader","slug":"failed-to-write-copy-header-e","errorCode":null,"errorMessage":"Failed to write COPY header: {e}","messagePattern":"Failed to write COPY header: (.+?)","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"crates/adapters/blockchain/src/cache/copy.rs","lineNumber":328,"sourceCode":"    /// - 11-byte signature: \"PGCOPY\\n\\xff\\r\\n\\0\"\n    /// - 4-byte flags field (all zeros)\n    /// - 4-byte header extension length (all zeros)\n    async fn write_copy_header(\n        &self,\n        copy_in: &mut sqlx::postgres::PgCopyIn<sqlx::pool::PoolConnection<sqlx::Postgres>>,\n    ) -> anyhow::Result<()> {\n        use std::io::Write;\n        let mut header = Vec::new();\n\n        // PostgreSQL binary copy header\n        header.write_all(b\"PGCOPY\\n\\xff\\r\\n\\0\")?; // Signature\n        header.write_all(&[0, 0, 0, 0])?; // Flags field\n        header.write_all(&[0, 0, 0, 0])?; // Header extension length\n\n        copy_in\n            .send(header)\n            .await\n            .map_err(|e| anyhow::anyhow!(\"Failed to write COPY header: {e}\"))?;\n        Ok(())\n    }\n\n    /// Writes a single block in PostgreSQL binary format.\n    ///\n    /// Each row in binary format consists of:\n    /// - 2-byte field count\n    /// - For each field: 4-byte length followed by data (or -1 for NULL)\n    async fn write_block_binary(\n        &self,\n        copy_in: &mut sqlx::postgres::PgCopyIn<sqlx::pool::PoolConnection<sqlx::Postgres>>,\n        chain_id: u32,\n        block: &Block,\n    ) -> anyhow::Result<()> {\n        use std::io::Write;\n        let mut row_data = Vec::new();\n\n        // Number of fields (14)","sourceCodeStart":310,"sourceCodeEnd":346,"githubUrl":"https://github.com/nautechsystems/nautilus_trader/blob/18893faf8b356be3320add8de2f861b0b647cf06/crates/adapters/blockchain/src/cache/copy.rs#L310-L346","documentation":"write_copy_header sends the 11-byte PostgreSQL COPY BINARY signature, flags, and header-extension-length fields via PgCopyIn::send; failure is wrapped in this error. If the header can't be written the whole COPY stream is unusable and the operation aborts. This is almost always a transport-level failure on the connection, not a data problem.","triggerScenarios":"The underlying connection was closed/reset between copy_in_raw and the header send (DB restart, idle timeout, network drop), the server aborted the COPY session, or the driver returned a protocol error.","commonSituations":"Long-lived PgPool connections killed by a firewall/LB idle timeout; Docker container DB restarted mid-run; pgbouncer recycling connections; running a very long job whose first write happens after an idle gap.","solutions":["Retry the entire copy_* call — the stream is unusable and must be restarted from copy_in_raw","Enable PgPoolOptions::test_before_acquire(true) or set a lower max_idle_timeout/max_lifetime","Check network path (firewall, LB idle timeouts, pgbouncer server_reset_query)","Confirm the DB server did not restart (check server logs) before re-running"],"exampleFix":"// before\nlet pool = PgPoolOptions::new().connect(url).await?;\n// after\nlet pool = PgPoolOptions::new().test_before_acquire(true).idle_timeout(Some(Duration::from_secs(60))).connect(url).await?;","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"for attempt in 1..=3 {\n    match handler.copy_blocks(chain_id, &blocks).await {\n        Ok(()) => break,\n        Err(e) if attempt < 3 && e.to_string().contains(\"Failed to write COPY header\") => {\n            tokio::time::sleep(Duration::from_secs(2u64.pow(attempt))).await;\n        }\n        Err(e) => return Err(e),\n    }\n}","preventionTips":["Set PgPoolOptions::test_before_acquire(true) and idle_timeout","Keep the DB connection active during long jobs (avoid long idle gaps before first write)","Check firewall/LB idle timeouts on the DB path","Retry the full copy_* call; a failed header invalidates the stream"],"tags":["postgres","copy-binary","broken-pipe"],"backgroundTag":"broken-pipe","analyzedSha":"18893faf8b356be3320add8de2f861b0b647cf06","analyzedAt":"2026-09-08T20:49:34.690Z","contentChangedAt":"2026-09-08T20:49:34.690Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}