apache/druid · error · IllegalStateException

No file present at the location [%s]. Unable to read the out

Error message

No file present at the location [%s]. Unable to read the outputs of stage [%d], partition [%d] for the worker [%d]

What it means

findSuccessfulPartitionOutput throws this ISE when the worker's '_success' marker file (got from getWorkerOutputSuccessFilePath) is absent in deep storage. The success file records which task produced the partition output; without it the consumer cannot resolve the output. Effectively the producer never signaled successful completion for this stage/partition.

Source

Thrown at multi-stage-query/src/main/java/org/apache/druid/msq/shuffle/input/DurableStorageInputChannelFactory.java:172

  }

  /**
   * Given an input worker number, stage number and the partition number, this method figures out the exact location
   * where the outputs would be present in the durable storage and returns the complete path or throws an exception
   * if no such file exists in the durable storage
   * More information at {@link DurableStorageOutputChannelFactory#createSuccessFile(String)}
   */
  public String findSuccessfulPartitionOutput(
      final String controllerTaskId,
      final int workerNo,
      final int stageNumber,
      final int partitionNumber
  ) throws IOException
  {
    String successfulFilePath = getWorkerOutputSuccessFilePath(controllerTaskId, stageNumber, workerNo);

    if (!storageConnector.pathExists(successfulFilePath)) {
      throw new ISE(
          "No file present at the location [%s]. Unable to read the outputs of stage [%d], partition [%d] for the worker [%d]",
          successfulFilePath,
          stageNumber,
          partitionNumber,
          workerNo
      );
    }

    String successfulTaskId;

    try (InputStream is = storageConnector.read(successfulFilePath)) {
      successfulTaskId = IOUtils.toString(is, StandardCharsets.UTF_8);
    }
    if (successfulTaskId == null) {
      throw new ISE("Unable to read the task id from the file: [%s]", successfulFilePath);
    }
    LOG.debug(
        "Reading output of stage [%d], partition [%d] from task id [%s]",

View on GitHub (pinned to 9b90983fd2)

Solutions

Check the producing worker task status/logs to see if it completed and wrote the success file.
Confirm the durable-storage location and prefix configs match across services so the same path is read and written.
Retry the query; if outputs were cleaned up, reduce cleanup interval or finish queries before cleanup.
If worker crashes recur, investigate OOM/kill issues on the task runner.

Example fix

// before: assuming success file exists
String successfulFilePath = getWorkerOutputSuccessFilePath(controllerTaskId, stageNumber, workerNo);
InputStream is = storageConnector.read(successfulFilePath);

// after: validate presence and fail with actionable context
if (!storageConnector.pathExists(successfulFilePath)) {
  throw new ISE("Success marker missing; did worker task %s fail? path=%s", workerNo, successfulFilePath);
}

Defensive patterns

Strategy: validation

Validate before calling

if (!storageConnector.pathExists(successMarkerPath)) {
  throw new IllegalStateException("Producer success marker absent — producer task likely failed: " + successMarkerPath);
}

Try / catch

try {
  fetchResults();
} catch (IllegalStateException e) {
  if (e.getMessage().contains("No file present at the location")) {
    checkOverlordTaskStatus(controllerTaskId); // surface producer failure to user
  }
  throw e;
}

Prevention

Check the producing task's status in the overlord before troubleshooting deep storage.
Set durable-storage cleanup intervals longer than your longest query.
Ensure consistent druid.storage config on all services.

When it happens

Trigger: remotePartitionPath -> findSuccessfulPartitionOutput runs pathExists on the success file and it is missing: producer task crashed before writing the marker, outputs cleaned up by retention, or wrong storage prefix configured so the marker is looked up in the wrong location.

Common situations: Query failure after a worker was killed/OOM-killed mid-stage; durable storage cleanup interval expiring while query still running; a bug/misconfig where overlord and task use different druid.storage prefixes.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/fce9adc824dabbea. Report an issue: GitHub.