{"record":{"id":"79c75db46d2d26eb","repo":"BoundaryML/baml","slug":"flush-timed-out-after","errorCode":null,"errorMessage":"Flush timed out after {:?}","messagePattern":"Flush timed out after (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"engine/baml-runtime/src/tracingv2/publisher/publisher.rs","lineNumber":1163,"sourceCode":"    log::debug!(\"Flushing traces [rust]\");\n    // Set a timeout to avoid waiting indefinitely.\n    let timeout_duration = Duration::from_secs(30);\n\n    // First try to flush the trace publisher (which should also flush blobs internally)\n    let mut publisher_result: Option<anyhow::Result<()>> = None;\n    if let Some(channel) = ensure_publisher_started() {\n        let (ack_tx, ack_rx) = tokio::sync::oneshot::channel();\n        let send_res = channel\n            .send(PublisherMessage::Flush(ack_tx))\n            .await\n            .map_err(|e| anyhow::anyhow!(e.to_string()));\n        if let Err(e) = send_res {\n            publisher_result = Some(Err(e));\n        } else {\n            publisher_result = Some(match timeout(timeout_duration, ack_rx).await {\n                Ok(Ok(())) => Ok(()),\n                Ok(Err(e)) => Err(e.into()),\n                Err(_) => Err(anyhow::anyhow!(\n                    \"Flush timed out after {:?}\",\n                    timeout_duration\n                )),\n            });\n        }\n    } else {\n        log::debug!(\"No publish channel found [rust]\");\n    }\n\n    // Always flush the blob uploader explicitly as well to guarantee no leftovers\n    log::debug!(\"Flushing blob uploader [rust]\");\n    let blob_result = flush_blob_uploader_channel(timeout_duration).await;\n    log::debug!(\"Flushing blob uploader [rust] completed\");\n    // Prefer reporting blob uploader errors if any; otherwise propagate publisher errors\n    blob_result?;\n\n    if let Some(Err(e)) = publisher_result {\n        return Err(e);","sourceCodeStart":1145,"sourceCodeEnd":1181,"githubUrl":"https://github.com/BoundaryML/baml/blob/bd85ce9dee1463ff04d27efd20531013a4ff46c1/engine/baml-runtime/src/tracingv2/publisher/publisher.rs#L1145-L1181","documentation":"Raised in flush (public) when waiting for the trace publisher worker's completion ack (ack_rx) exceeds timeout_duration. flush() sends a sentinel and waits for all queued batches to be published; this error means the worker did not finish draining the queue in time, so some trace events may not have reached the collector.","triggerScenarios":"flush() called while the publisher still holds up to 4 full batches of events and the network upload is slower than the timeout: slow collector responses, high trace volume, or a timeout set too low (e.g. seconds before process exit).","commonSituations":"Flushing at process shutdown with a large accumulated trace backlog, slow/unresponsive BAML collector, brief network outage during flush, or a very short flush timeout in serverless/Lambda environments.","solutions":["Increase the flush timeout passed to flush().","Flush periodically during execution instead of once with a big backlog at exit.","Verify collector latency/availability; retry flush after a backoff.","Reduce trace volume (BAML_TRACE_BATCH_SIZE / fewer events) if backlog is chronic."],"exampleFix":"// before\nawait publisher.flush(5)\n// after\nawait publisher.flush(60)  # allow queued batches to drain on slow links","handlingStrategy":"try-catch","validationCode":"// estimate needed timeout from backlog\nconst BATCH: usize = 50;\nlet est = Duration::from_secs((publisher.pending_events() / BATCH + 2) as u64);\nlet timeout = est.max(Duration::from_secs(30));","typeGuard":"fn is_flush_timeout(e: &anyhow::Error) -> bool {\n    e.to_string().starts_with(\"Flush timed out\")\n}","tryCatchPattern":"match publisher.flush(Duration::from_secs(60)).await {\n    Err(e) if e.to_string().starts_with(\"Flush timed out\") => {\n        log::warn!(\"flush incomplete; some trace events may be lost\");\n        // optionally retry once with a longer timeout\n    }\n    Err(e) => log::error!(\"flush failed: {e}\"),\n    Ok(_) => {}\n}","preventionTips":["Use generous timeouts (>=30s) for shutdown flushes","Flush periodically during long-running jobs to limit backlog","Tune BAML_TRACE_BATCH_SIZE for your trace volume","Monitor collector latency; retry flush on timeout"],"tags":["timeout","async","baml","tracing"],"backgroundTag":"request-timeout","analyzedSha":"bd85ce9dee1463ff04d27efd20531013a4ff46c1","analyzedAt":"2026-09-12T03:38:25.718Z","contentChangedAt":"2026-09-12T03:38:25.718Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}