denoland/deno · error
OTEL export timed out after {}ms
Error message
OTEL export timed out after {}ms What it means
OTLP-over-gRPC export path in Deno's OpenTelemetry runtime: HyperClient::grpc_request wraps the whole request (connect + response + body + trailers, capped at 1 MiB) in tokio::time::timeout(self.timeout). If the collector does not complete the round trip within the budget, the future is dropped and ErrorKind::TimedOut with this message is returned. The timeout comes from OTEL_EXPORTER_OTLP_TIMEOUT in milliseconds (default 10000).
Source
Thrown at ext/telemetry/lib.rs:919
request: Request<Vec<u8>>,
) -> Result<
(hyper::http::response::Parts, Option<hyper::HeaderMap>),
Box<dyn std::error::Error + Send + Sync>,
> {
let (parts, body) = request.into_parts();
let request = Request::from_parts(parts, Full::from(body));
let result = tokio::time::timeout(self.timeout, async {
let response = self.inner.request(request).await?;
let (parts, body) = response.into_parts();
let collected = http_body_util::Limited::new(body, 1024 * 1024)
.collect()
.await?;
let trailers = collected.trailers().cloned();
Ok::<_, Box<dyn std::error::Error + Send + Sync>>((parts, trailers))
})
.await
.map_err(|_| -> Box<dyn std::error::Error + Send + Sync> {
Box::new(std::io::Error::new(
std::io::ErrorKind::TimedOut,
format!("OTEL export timed out after {}ms", self.timeout.as_millis()),
))
})??;
Ok(result)
}
}
#[async_trait::async_trait]
impl opentelemetry_http::HttpClient for HyperClient {
async fn send_bytes(
&self,
request: Request<Bytes>,
) -> Result<Response<Bytes>, HttpError> {
let (parts, body) = request.into_parts();
let request = Request::from_parts(parts, Full::new(body));
let response = tokio::time::timeout(self.timeout, async {
let response = self.inner.request(request).await?;View on GitHub (pinned to 9ad36f7a2c)
Solutions
- Raise the budget: export OTEL_EXPORTER_OTLP_TIMEOUT=30000 (milliseconds).
- Verify the endpoint is reachable from the app container: curl http://<collector>:4317/ and check DNS/firewall; for gRPC use port 4317.
- Run/health-check a local collector sidecar and point OTEL_EXPORTER_OTLP_ENDPOINT at 127.0.0.1 so only the collector handles the slow WAN hop.
- Reduce export size/frequency (OTEL_BSP_* batch settings) so a round trip fits the budget.
Example fix
# before export OTEL_EXPORTER_OTLP_ENDPOINT=https://otel.example.com:4317 deno run --unstable-otel app.ts # exports time out after 10000ms # after export OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:4317 # local sidecar export OTEL_EXPORTER_OTLP_TIMEOUT=30000 # 30s budget deno run --unstable-otel app.ts
Defensive patterns
Strategy: retry
Validate before calling
const ep = process.env.OTEL_EXPORTER_OTLP_ENDPOINT ?? "http://127.0.0.1:4317";
const ok = await fetch(ep, { signal: AbortSignal.timeout(2_000) }).then(r => true, () => false);
if (!ok) console.warn(`OTLP collector not reachable at ${ep} — exports will time out`); Try / catch
try { await exportOnce(); } catch (e) { if (e instanceof Error && /OTEL export timed out/.test(e.message)) console.warn("OTLP gRPC export timed out — batch dropped; check collector 4317 and OTEL_EXPORTER_OTLP_TIMEOUT"); else throw e; } Prevention
- Run the collector as a local sidecar (127.0.0.1:4317) and let it own WAN retries.
- Set OTEL_EXPORTER_OTLP_TIMEOUT explicitly (milliseconds) for high-latency environments.
- Add collector readiness checks before starting instrumented workloads.
- Note that OTel batch processors drop failed batches after retries — monitor collector health, don't rely on the app.
When it happens
Trigger: deno run -- unstable-otel with OTEL_EXPORTER_OTLP_PROTOCOL=grpc and a collector endpoint that is unreachable, slow, or stalling (connection hang, TLS handshake slowness, collector overloaded) so the export exceeds OTEL_EXPORTER_OTLP_TIMEOUT.
Common situations: Remote collector behind slow links or flaky DNS; OTEL collector sidecar not yet up when the app starts; batch exports larger than the link can push in 10s; firewalls dropping packets (black hole) instead of refusing; default timeout too small for high-latency environments.
Understand the failure class
Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.
- Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.
Related errors
- invalid vsock addr
- Env var OTEL_EXPORTER_OTLP_PROTOCOL specifies an unsupported
- TracerProvider can not be constructed
- Bench attempted to exit with exit code: ${exitCode}
- Unsupported 'certFile' / 'keyFile' options provided: use 'ce
AI-assisted analysis of denoland/deno@9ad36f7a2c (2026-08-20).
Data as JSON: /api/errors/ec985d7f4e4c7a1c.
Report an issue: GitHub.