neondatabase/neon · error · DownloadError

GCS GET response contained no response body

Error message

GCS GET response contained no response body

What it means

During download, the first request (GET object metadata as JSON) returned a non-success status other than 404. Like the head path, the message is generic ('no response body') while the true cause is the discarded status code — auth failure, throttling, or a server error on the metadata GET.

Source

Thrown at libs/remote_storage/src/gcs_bucket.rs:736

                    .map_err(|e: gcp_auth::Error| DownloadError::Other(e.into()))?
                    .as_str(),
            )
            .send();

        let obj_metadata = tokio::select! {
            res = res => res,
            _ = tokio::time::sleep(self.timeout) => return Err(DownloadError::Timeout),
            _ = cancel.cancelled() => return Err(DownloadError::Cancelled),
        };

        let resp = match obj_metadata {
            Ok(resp) => {
                if !resp.status().is_success() {
                    match resp.status() {
                        StatusCode::NOT_FOUND => return Err(DownloadError::NotFound),
                        _ => {
                            return Err(DownloadError::Other(anyhow::anyhow!(
                                "GCS GET response contained no response body"
                            )));
                        }
                    }
                } else {
                    resp
                }
            }
            _ => {
                return Err(DownloadError::Other(anyhow::anyhow!("download gcs object")));
            }
        };

        let body = resp
            .text()
            .await
            .map_err(|e: reqwest::Error| DownloadError::Other(e.into()))?;

        let resp: GCSObject = serde_json::from_str(&body)

View on GitHub (pinned to 8f60b04da4)

Solutions

1. Log the status code (patch the match arm or enable debug tracing) to classify auth vs throttle vs server error
2. Refresh credentials and retry the download
3. Backoff-retry on 429/5xx; fix permissions on 403
4. Check GCP status dashboard if 5xx persists cluster-wide

Example fix

// before: status discarded, error is opaque
_ => return Err(DownloadError::Other(anyhow::anyhow!("GCS GET response contained no response body"))),

// after: include the status in the error
status => return Err(DownloadError::Other(anyhow::anyhow!("GCS metadata GET failed: {}", status))),

// Download-time metadata failures: bounded retry, then propagate.
async fn download_with_retry(storage: &Arc<GenericRemoteStorage>, from: &RemotePath, cancel: &CancellationToken) -> Result<Download, DownloadError> {
    let mut attempt = 0;
    loop {
        attempt += 1;
        match storage.download(from, &cancel).await {
            Ok(dl) => return Ok(dl),
            Err(DownloadError::NotFound) => return Err(DownloadError::NotFound),
            Err(DownloadError::Other(e)) if attempt < 3 => {
                tracing::warn!("GCS metadata GET failed (attempt {attempt}): {e:#}");
                tokio::time::sleep(Duration::from_millis(300 * attempt as u64)).await;
            }
            Err(e) => return Err(e),
        }
    }
}

When it happens

Trigger: download() when the object metadata GET returns 401/403 (token expired between listing and download), 429/529, or 5xx — anything except 200 and 404.

Common situations: Tokens expiring mid-operation under load; GCS throttling when many downloads start simultaneously (permit-concurrency bursts); transient 5xx.

Related errors

• GCS head response contained no response body
• GCS PUT error \n\t {:?}
• no items returned
• no 'updated' field
• max keys reached

AI-assisted analysis of neondatabase/neon@8f60b04da4 (2026-08-16). Data as JSON: /api/errors/b40b5c7d109aab50. Report an issue: GitHub.