{"record":{"id":"a2232b86814dc4fd","repo":"neondatabase/neon","slug":"gcs-head-response-contained-no-response-body","errorCode":null,"errorMessage":"GCS head response contained no response body","messagePattern":"GCS head response contained no response body","errorType":"exception","errorClass":"DownloadError","httpStatus":null,"severity":"error","filePath":"libs/remote_storage/src/gcs_bucket.rs","lineNumber":646,"sourceCode":"            _ = cancel.cancelled() => return Err(TimeoutOrCancel::Cancel.into()),\n        };\n\n        let res = res.map_err(|_e| DownloadError::Timeout)?;\n\n        // do not incl. timeouts as errors in metrics but cancellations\n        let started_at = ScopeGuard::into_inner(started_at);\n        crate::metrics::BUCKET_METRICS\n            .req_seconds\n            .observe_elapsed(kind, &res, started_at);\n\n        let data = match res {\n            Ok(data) => {\n                if !data.status().is_success() {\n                    match data.status() {\n                        StatusCode::NOT_FOUND => return Err(DownloadError::NotFound),\n                        _ => {\n                            return Err(DownloadError::Other(anyhow::anyhow!(\n                                \"GCS head response contained no response body\"\n                            )));\n                        }\n                    }\n                } else {\n                    data\n                }\n            }\n            Err(e) => {\n                crate::metrics::BUCKET_METRICS.req_seconds.observe_elapsed(\n                    kind,\n                    AttemptOutcome::Err,\n                    started_at,\n                );\n\n                return Err(DownloadError::Other(\n                    anyhow::Error::new(e).context(\"error in HEAD of GCS object\"),\n                ));\n            }","sourceCodeStart":628,"sourceCodeEnd":664,"githubUrl":"https://github.com/neondatabase/neon/blob/8f60b04da47ffefe0e52bda2440134b42874eb75/libs/remote_storage/src/gcs_bucket.rs#L628-L664","documentation":"The GCS object metadata (HEAD-style) request returned a non-success status other than 404. Because a metadata response carries no body to include, the code reports a generic 'no response body' message; the real classification is the HTTP status, which the catch-all match arm discards. Auth (401/403), throttling (429/529), and server errors (5xx) all surface identically here.","triggerScenarios":"head_object receiving 401/403 (expired token or missing storage.objects.get), 429/529 (rate limiting), or 5xx — anything except 200 and 404.","commonSituations":"Expired gcp_auth token in a long-lived process; missing IAM permission; GCS throttling under heavy listing/probing load; transient 5xx during GCP incidents.","solutions":["Capture the actual status before the match discards it — patch or debug-log data.status() so the failure is classifiable","Refresh credentials and retry","Apply backoff retry for 429/5xx statuses","Check IAM: storage.objects.get on the bucket for the identity in use"],"exampleFix":"// before: every non-404 failure is an opaque error\nmatch data.status() {\n    StatusCode::NOT_FOUND => return Err(DownloadError::NotFound),\n    _ => return Err(DownloadError::Other(anyhow::anyhow!(\"GCS head response contained no response body\"))),\n}\n\n// after: carry the status so callers can retry intelligently\nmatch data.status() {\n    StatusCode::NOT_FOUND => return Err(DownloadError::NotFound),\n    status => return Err(DownloadError::Other(anyhow::anyhow!(\"GCS head failed: {}\", status))),\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"// Generic message hides the status: retry transient-looking head failures, surface the rest.\nasync fn head_with_retry(storage: &Arc<GenericRemoteStorage>, key: String, cancel: &CancellationToken) -> Result<ListingObject, DownloadError> {\n    let mut attempt = 0;\n    loop {\n        attempt += 1;\n        match storage.head_object(key.clone(), cancel).await {\n            Ok(obj) => return Ok(obj),\n            Err(DownloadError::NotFound) => return Err(DownloadError::NotFound),\n            Err(DownloadError::Other(e)) if attempt < 3 => {\n                tracing::warn!(\"GCS head failed (attempt {attempt}): {e:#}\");\n                tokio::time::sleep(Duration::from_millis(300 * attempt as u64)).await;\n            }\n            Err(e) => return Err(e),\n        }\n    }\n}","preventionTips":["Treat any GCS metadata failure other than NotFound as retryable-once — most are throttle/transient","Refresh tokens on a schedule shorter than their lifetime","Push for status codes in error messages upstream; opaque messages cost real debugging time"],"tags":["gcs","google-cloud-storage","head-request","http-status","observability"],"backgroundTag":"http-error-response","analyzedSha":"8f60b04da47ffefe0e52bda2440134b42874eb75","analyzedAt":"2026-08-16T23:39:28.135Z","schemaVersion":2},"datasetVersion":"2026-08-17T04:17:16.089Z"}