{"record":{"id":"89d4abed8a2afa32","repo":"neondatabase/neon","slug":"missing-etag-header-89d4ab","errorCode":null,"errorMessage":"Missing ETag header","messagePattern":"Missing ETag header","errorType":"exception","errorClass":"DownloadError","httpStatus":null,"severity":"error","filePath":"libs/remote_storage/src/s3_bucket.rs","lineNumber":318,"sourceCode":"                    kind,\n                    AttemptOutcome::Err,\n                    started_at,\n                );\n\n                return Err(DownloadError::Other(\n                    anyhow::Error::new(e).context(\"download s3 object\"),\n                ));\n            }\n        };\n\n        // even if we would have no timeout left, continue anyways. the caller can decide to ignore\n        // the errors considering timeouts and cancellation.\n        let remaining = self.timeout.saturating_sub(started_at.elapsed());\n\n        let metadata = object_output.metadata().cloned().map(StorageMetadata);\n        let etag = object_output\n            .e_tag\n            .ok_or(DownloadError::Other(anyhow::anyhow!(\"Missing ETag header\")))?\n            .into();\n        let last_modified = object_output\n            .last_modified\n            .ok_or(DownloadError::Other(anyhow::anyhow!(\n                \"Missing LastModified header\"\n            )))?\n            .try_into()\n            .map_err(|e: ConversionError| DownloadError::Other(e.into()))?;\n\n        let body = object_output.body;\n        let body = ByteStreamAsStream::from(body);\n        let body = PermitCarrying::new(permit, body);\n        let body = TimedDownload::new(started_at, body);\n\n        let cancel_or_timeout = crate::support::cancel_or_timeout(remaining, cancel.clone());\n        let body = crate::support::DownloadStream::new(cancel_or_timeout, body);\n\n        Ok(Download {","sourceCodeStart":300,"sourceCodeEnd":336,"githubUrl":"https://github.com/neondatabase/neon/blob/8f60b04da47ffefe0e52bda2440134b42874eb75/libs/remote_storage/src/s3_bucket.rs#L300-L336","documentation":"Thrown by S3Bucket's download path after a successful GetObject when the S3 response carries no ETag header. Real AWS S3 (and compliant servers like MinIO) always set ETag on GetObject, so this almost always means the configured endpoint is a partially S3-compatible emulator or a proxy that strips headers. The ETag is required because the returned Download struct exposes etag for later conditional requests (if_none_match).","triggerScenarios":"Calling GenericRemoteStorage::download / download_object (e.g. a pageserver fetching a layer or timeline index) against an S3-compatible endpoint whose GetObject response omits the ETag header, or through a proxy that removes response headers.","commonSituations":"Local dev/CI against a hand-rolled mock or partially compliant S3 emulator; a corporate proxy or gateway stripping headers; an S3-like provider that skips ETag for some object classes.","solutions":["Point dev/test at an endpoint that returns ETag (real S3 or MinIO)","Verify the endpoint directly: aws s3api get-object --bucket <b> --key <k> /dev/null --query ETag","Fix or upgrade the emulator/gateway to include ETag in GetObject responses","If you control the server, set the ETag header on all GetObject responses"],"exampleFix":"// before: dev/test against an emulator that omits ETag\nremote_storage_config = { bucket = \"local\", endpoint = \"http://mock:5000\" }\n// after: use a compliant S3 server\nremote_storage_config = { bucket = \"local\", endpoint = \"http://minio:9000\" }","handlingStrategy":"try-catch","validationCode":"use aws_sdk_s3::Client;\n\nasync fn endpoint_returns_etag(client: &Client, bucket: &str, key: &str) -> anyhow::Result<bool> {\n    let head = client.head_object().bucket(bucket).key(key).send().await?;\n    Ok(head.e_tag.is_some())\n}","typeGuard":"fn is_missing_etag(err: &remote_storage::DownloadError) -> bool {\n    matches!(err, remote_storage::DownloadError::Other(e)\n        if e.to_string().contains(\"Missing ETag header\"))\n}","tryCatchPattern":"match storage.download(&path, &cancel).await {\n    Err(DownloadError::Other(e)) if e.to_string().contains(\"Missing ETag header\") => {\n        // endpoint incompatibility: do not retry; report the endpoint configuration\n    }\n    Err(DownloadError::NotFound) => { /* object absent */ }\n    other => other?,\n}","preventionTips":["Use real S3 or MinIO in integration tests","Add a startup probe that HEADs a known object and asserts ETag presence","Alert on this exact message: it signals protocol non-compliance, not a transient failure"],"tags":["s3","rust","aws-sdk","remote-storage","http-header"],"backgroundTag":"missing-http-response-header","analyzedSha":"8f60b04da47ffefe0e52bda2440134b42874eb75","analyzedAt":"2026-08-16T23:39:28.135Z","schemaVersion":2},"datasetVersion":"2026-08-17T04:17:16.089Z"}