{"record":{"id":"7249cf14d96aa782","repo":"apache/druid","slug":"timed-out-after-d-millis-during-flushing","errorCode":null,"errorMessage":"Timed out after [%d] millis during flushing","messagePattern":"Timed out after \\[(.+?)\\] millis during flushing","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/java/util/emitter/core/HttpPostEmitter.java","lineNumber":429,"sourceCode":"      flush((Batch) batchObj);\n    }\n  }\n\n  private void flush(Batch batch) throws IOException\n  {\n    if (batch == null) {\n      return;\n    }\n    batch.seal();\n    try {\n      // This check doesn't always awaits for this exact batch to be emitted, because another batch could be dropped\n      // from the queue ahead of this one, in limitBuffersToEmitSize(). But there is no better way currently to wait for\n      // the exact batch, and it's not that important.\n      emittedBatchCounter.awaitCount(batch.batchNumber, config.getFlushTimeOut(), TimeUnit.MILLISECONDS);\n    }\n    catch (TimeoutException e) {\n      String message = StringUtils.format(\"Timed out after [%d] millis during flushing\", config.getFlushTimeOut());\n      throw new IOException(message, e);\n    }\n    catch (InterruptedException e) {\n      log.debug(\"Thread Interrupted\");\n      Thread.currentThread().interrupt();\n      throw new IOException(\"Thread Interrupted while flushing\", e);\n    }\n  }\n\n  @Override\n  @LifecycleStop\n  public void close() throws IOException\n  {\n    synchronized (startLock) {\n      if (running) {\n        running = false;\n        Object lastBatch = concurrentBatch.getAndSet(null);\n        if (lastBatch instanceof Batch) {\n          flush((Batch) lastBatch);","sourceCodeStart":411,"sourceCodeEnd":447,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/java/util/emitter/core/HttpPostEmitter.java#L411-L447","documentation":"flush() waits (emittedBatchCounter.awaitCount) until the batch being flushed has actually been sent by the EmittingThread. If the batch is not emitted within HttpEmitterConfig.flushTimeOut milliseconds, a TimeoutException is converted to IOException('Timed out after [%d] millis during flushing'). This indicates the sending pipeline (network, remote endpoint, queue backlog) is stalled.","triggerScenarios":"flush() or close() while the EmittingThread is stuck posting large batches to a slow/unreachable recipient; flushTimeOut too small for batch size/network latency; event queue backlog larger than the thread can drain in time.","commonSituations":"Remote telemetry endpoint (e.g. overlord or external HTTP service) hanging or rate-limiting; network partitions; misconfigured flushTimeOut (default a few seconds) with big maxBatchSize; calling close() during shutdown with many queued events.","solutions":["Increase HttpEmitterConfig.flushTimeOut to cover worst-case send duration.","Verify the recipientBaseUrl endpoint is reachable and responding; check network/proxy issues.","Reduce maxBatchSize or batchQueueSize so individual sends complete faster.","Catch IOException from flush()/close() and log; consider retrying flush after checking emitter health."],"exampleFix":"// before\nHttpEmitterConfig.builder().setFlushTimeOut(1000).build();\n// after\nHttpEmitterConfig.builder().setFlushTimeOut(60000).build();","handlingStrategy":"try-catch","validationCode":"// pre-check endpoint reachability\nHttpURLConnection c = (HttpURLConnection) new URI(config.getRecipientBaseUrl()).toURL().openConnection();\nc.setConnectTimeout(2000);\nif (c.getResponseCode() >= 400) log.warn(\"Recipient unhealthy\");","typeGuard":null,"tryCatchPattern":"try {\n  emitter.flush();\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"Timed out\")) {\n    log.error(e, \"Flush timed out; recipient slow or batch too large\");\n  }\n}","preventionTips":["Set flushTimeOut generously for batch size and network RTT","Monitor recipient endpoint latency","Keep maxBatchSize within what the endpoint drains quickly"],"tags":["timeout","flush","network","emitter"],"backgroundTag":"request-timeout","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}