{"record":{"id":"6fe2c5e634c79250","repo":"neondatabase/neon","slug":"received-abort-for-copy-from-from-to-to","errorCode":null,"errorMessage":"Received abort for copy from {from} to {to}.","messagePattern":"Received abort for copy from (.+?) to (.+?)\\.","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"libs/remote_storage/src/azure_blob.rs","lineNumber":903,"sourceCode":"        let op = async {\n            let blob_client = self.client.blob_client(self.relative_path_to_name(to));\n\n            let source_url = format!(\n                \"{}/{}\",\n                self.client.url()?,\n                self.relative_path_to_name(from)\n            );\n\n            let builder = blob_client.copy(Url::from_str(&source_url)?);\n            let copy = builder.into_future();\n\n            let result = copy.await?;\n\n            copy_status = Some(result.copy_status);\n            loop {\n                match copy_status.as_ref().expect(\"we always set it to Some\") {\n                    CopyStatus::Aborted => {\n                        anyhow::bail!(\"Received abort for copy from {from} to {to}.\");\n                    }\n                    CopyStatus::Failed => {\n                        anyhow::bail!(\"Received failure response for copy from {from} to {to}.\");\n                    }\n                    CopyStatus::Success => return Ok(()),\n                    CopyStatus::Pending => (),\n                }\n                // The copy is taking longer. Waiting a second and then re-trying.\n                // TODO estimate time based on copy_progress and adjust time based on that\n                tokio::time::sleep(Duration::from_millis(1000)).await;\n                let properties = blob_client.get_properties().into_future().await?;\n                let Some(status) = properties.blob.properties.copy_status else {\n                    tracing::warn!(\"copy_status for copy is None!, from={from}, to={to}\");\n                    return Ok(());\n                };\n                copy_status = Some(status);\n            }\n        };","sourceCodeStart":885,"sourceCodeEnd":921,"githubUrl":"https://github.com/neondatabase/neon/blob/8f60b04da47ffefe0e52bda2440134b42874eb75/libs/remote_storage/src/azure_blob.rs#L885-L921","documentation":"Azure's Copy Blob is asynchronous: the backend polls get_properties once per second and matches the reported copy_status. CopyStatus::Aborted means Azure terminated the pending copy — most commonly because the destination blob was modified or overwritten while the copy was pending, the copy was explicitly aborted, or the source became unavailable.","triggerScenarios":"Calling RemoteStorage::copy on Azure while another writer overwrites/puts the destination blob during the pending copy; another actor issues Abort Copy Blob; the source blob is deleted mid-copy; two concurrent copies race to the same destination key.","commonSituations":"Concurrent upload and copy targeting the same destination path; retry storms where a retried copy races the previous attempt; GC deleting source timelines while copies are in flight.","solutions":["Serialize writers: ensure only one upload/copy targets the destination key at a time","Retry the copy once the conflicting writer has finished","Verify the source blob still exists immediately before issuing the copy","For small objects use download+upload instead of server-side copy to avoid long-pending async copies"],"exampleFix":"// before: concurrent writers can abort the pending copy\nlet _ = storage.copy(&from, &to, &cancel).await;\n\n// after: single-writer per destination via a keyed lock\nlet _guard = key_locks.lock(to.clone()).await;\nstorage.copy(&from, &to, &cancel).await?;","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"// Catch, distinguish abort from other failures, re-issue the copy after the contended writer settles.\nmatch storage.copy(&from, &to, &cancel).await {\n    Ok(()) => {}\n    Err(e) if format!(\"{e:#}\").contains(\"Received abort for copy\") => {\n        tracing::warn!(\"copy aborted (contended destination), retrying: {from} -> {to}\");\n        tokio::time::sleep(Duration::from_secs(1)).await;\n        storage.copy(&from, &to, &cancel).await?;\n    }\n    Err(e) => return Err(e),\n}","preventionTips":["Serialize all writers (uploads and copies) per destination key with a keyed lock or queue","Never run two copies to the same destination concurrently, including across retry logic","Keep source objects alive (defer GC/retention deletion) until in-flight copies complete"],"tags":["azure","blob-storage","server-side-copy","concurrency"],"backgroundTag":"server-side-copy-aborted","analyzedSha":"8f60b04da47ffefe0e52bda2440134b42874eb75","analyzedAt":"2026-08-16T23:39:28.135Z","schemaVersion":2},"datasetVersion":"2026-08-17T04:17:16.089Z"}