{"record":{"id":"3f56b9f5c6b602ee","repo":"databendlabs/databend","slug":"desc","errorCode":null,"errorMessage":"{desc}","messagePattern":"\\{desc\\}","errorType":"exception","errorClass":"Unexpected","httpStatus":null,"severity":"error","filePath":"src/common/storage/src/http_client.rs","lineNumber":313,"sourceCode":"        mem::swap(hr.headers_mut().unwrap(), resp.headers_mut());\n\n        let bs = HttpBody::new(\n            resp.bytes_stream()\n                .try_filter(|v| future::ready(!v.is_empty()))\n                .map_ok(Buffer::from)\n                .map_err(move |err| {\n                    to_opendal_unexpected_error(err, &uri, \"read data from http response\")\n                }),\n            content_length,\n        );\n\n        let resp = hr.body(bs).expect(\"response must build succeed\");\n        Ok(resp)\n    }\n}\n\nfn to_opendal_unexpected_error(err: reqwest::Error, uri: &http::Uri, desc: &str) -> opendal::Error {\n    let mut oe = opendal::Error::new(opendal::ErrorKind::Unexpected, desc)\n        .with_operation(\"http_util::Client::send\")\n        .with_context(\"url\", uri.to_string());\n    if is_temporary_error(&err) {\n        oe = oe.set_temporary();\n    }\n    oe = oe.set_source(err);\n    oe\n}\n\n#[inline]\nfn is_temporary_error(err: &reqwest::Error) -> bool {\n    // error sending request\n    err.is_request() ||\n        // request or response body error\n        err.is_body() ||\n        // error decoding response body, for example, connection reset.\n        err.is_decode()\n}","sourceCodeStart":295,"sourceCodeEnd":331,"githubUrl":"https://github.com/databendlabs/databend/blob/288d84d76e20a2f8f7173bda9691eb6ece301aa9/src/common/storage/src/http_client.rs#L295-L331","documentation":"Built by `to_opendal_unexpected_error`, the common mapping of a failed `reqwest` send in `http_util::Client::fetch` into an `opendal::Error` of kind Unexpected. The error carries the operation name, the target URL, a human description, and the original reqwest error as its source; it is marked temporary when the failure looks transient (connection reset, timeout, 5xx-style conditions).","triggerScenarios":"Any `fetch` where the underlying reqwest request fails at the transport level: DNS failure, connection refused/reset, TLS handshake error, request timeout, or the body send/redirect machinery erroring — anything surfaced as `reqwest::Error`.","commonSituations":"Storage endpoint unreachable from the node (network partition, firewall, security group), DNS misconfiguration, expired/mismatched TLS certificates, or transient upstream outages during heavy load.","solutions":["Inspect `err.source()`/source chain and the `url` context to see whether it's DNS, connect, TLS, or timeout.","If the error is marked temporary, retry with backoff — the client already classified it as transient.","Verify network reachability to the endpoint from the node (curl the URL, check firewall/security-group rules).","Check TLS certificate validity on the endpoint if the source error is a handshake failure.","Correct the endpoint/URL in storage config if it points at a wrong host or port."],"exampleFix":"// before: unreachable endpoint\nlet client = HttpClient::new();\nlet resp = client.fetch(\"https://storage.internal:9443\", ...).await?;\n// after: correct, reachable endpoint\nlet resp = client.fetch(\"https://s3.us-east-1.amazonaws.com\", ...).await?;","handlingStrategy":"retry","validationCode":"fn endpoint_reachable(url: &str) -> bool {\n    std::net::TcpStream::connect((\n        url::Url::parse(url).ok().and_then(|u| u.host_str().map(String::from))?.as_str(),\n        url::Url::parse(url).ok().and_then(|u| u.port_or_known_default()).unwrap_or(443),\n    )).is_ok()\n}","typeGuard":"fn is_temporary_opendal_error(e: &opendal::Error) -> bool {\n    e.is_temporary()\n}","tryCatchPattern":"match client.fetch(url, range).await {\n    Ok(r) => Ok(r),\n    Err(e) if e.is_temporary() => retry_with_backoff(3, || client.fetch(url, range)).await,\n    Err(e) => Err(anyhow!(\"request to {url} failed permanently: {e}\")),\n}","preventionTips":["Check e.is_temporary() before retrying; only transient errors benefit.","Verify DNS and firewall/security-group rules allow egress to the storage endpoint.","Monitor TLS certificate expiry on storage endpoints.","Log the url context attached to the opendal error when triaging."],"tags":["http","network","opendal","reqwest","request-failed"],"backgroundTag":"http-request-failed","analyzedSha":"288d84d76e20a2f8f7173bda9691eb6ece301aa9","analyzedAt":"2026-09-11T11:29:36.208Z","contentChangedAt":"2026-09-11T11:29:36.208Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}