{"record":{"id":"380367b529099492","repo":"apache/druid","slug":"exception-while-waiting-to-extract-distributions","errorCode":null,"errorMessage":"Exception while waiting to extract distributions.","messagePattern":"Exception while waiting to extract distributions\\.","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"indexing-service/src/main/java/org/apache/druid/indexing/common/task/batch/parallel/PartialDimensionDistributionParallelIndexTaskRunner.java","lineNumber":253,"sourceCode":"  }\n\n  private File getDistributionJsonFile(File intervalDir, String subTaskId)\n  {\n    return new File(intervalDir, subTaskId);\n  }\n\n  /**\n   * Waits for distributions from pending reports (if any) to be extracted.\n   */\n  private void waitToProcessPendingReports()\n  {\n    log.info(\"Waiting to extract distributions from sub-task reports.\");\n    try {\n      allReportsProcessedPhaser.arriveAndAwaitAdvance();\n      executor.shutdownNow();\n    }\n    catch (Exception e) {\n      throw new ISE(e, \"Exception while waiting to extract distributions.\");\n    }\n  }\n\n  private File createDistributionsDir()\n  {\n    File taskTempDir = getToolbox().getConfig().getTaskTempDir(getTaskId());\n    File distributionsDir = new File(taskTempDir, \"dimension_distributions\");\n    try {\n      FileUtils.mkdirp(distributionsDir);\n      return distributionsDir;\n    }\n    catch (IOException e) {\n      throw new ISE(e, \"Could not create temp distribution directory.\");\n    }\n  }\n\n  private void cleanupDistributionsDir()\n  {","sourceCodeStart":235,"sourceCodeEnd":271,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/indexing-service/src/main/java/org/apache/druid/indexing/common/task/batch/parallel/PartialDimensionDistributionParallelIndexTaskRunner.java#L235-L271","documentation":"waitToProcessPendingReports() blocks on a Phaser until all pending sub-task reports have been processed by an executor, then shuts the executor down. If the await is interrupted or otherwise throws, the exception is wrapped in a generic ISE.","triggerScenarios":"A worker thread in the report-processing executor dies unexpectedly, or the awaiting thread is interrupted (task cancellation, supervisor shutdown) while allReportsProcessedPhaser.arriveAndAwaitAdvance() is in progress.","commonSituations":"Overlord/worker interruption during task kill; bugs in distribution extraction threads throwing before arriving at the phaser; JVM thread pool exhaustion on the Middle Manager.","solutions":["Look at the wrapped cause and earlier log lines from the report-processing runnable for the real failure.","Retry the batch task; phaser/interruption failures are usually transient.","If reproducible, check for executor/thread pool configuration issues on the indexing service worker."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"catch (IllegalStateException e) { if (e.getMessage().contains(\"Exception while waiting to extract distributions\")) { log.error(\"cause\", e.getCause()); retryTask(); } else { throw e; } }","preventionTips":["Avoid aggressive task-kill/interrupt during the distribution phase.","Size worker thread pools adequately for parallel batch ingestion.","Investigate the wrapped cause and prior logs for the underlying extraction failure."],"tags":["druid","concurrency","interruption","batch-ingestion"],"backgroundTag":"request-timeout","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}