{"record":{"id":"fce9adc824dabbea","repo":"apache/druid","slug":"no-file-present-at-the-location-s-unable-to-re","errorCode":null,"errorMessage":"No file present at the location [%s]. Unable to read the outputs of stage [%d], partition [%d] for the worker [%d]","messagePattern":"No file present at the location \\[(.+?)\\]\\. Unable to read the outputs of stage \\[(.+?)\\], partition \\[(.+?)\\] for the worker \\[(.+?)\\]","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"multi-stage-query/src/main/java/org/apache/druid/msq/shuffle/input/DurableStorageInputChannelFactory.java","lineNumber":172,"sourceCode":"  }\n\n  /**\n   * Given an input worker number, stage number and the partition number, this method figures out the exact location\n   * where the outputs would be present in the durable storage and returns the complete path or throws an exception\n   * if no such file exists in the durable storage\n   * More information at {@link DurableStorageOutputChannelFactory#createSuccessFile(String)}\n   */\n  public String findSuccessfulPartitionOutput(\n      final String controllerTaskId,\n      final int workerNo,\n      final int stageNumber,\n      final int partitionNumber\n  ) throws IOException\n  {\n    String successfulFilePath = getWorkerOutputSuccessFilePath(controllerTaskId, stageNumber, workerNo);\n\n    if (!storageConnector.pathExists(successfulFilePath)) {\n      throw new ISE(\n          \"No file present at the location [%s]. Unable to read the outputs of stage [%d], partition [%d] for the worker [%d]\",\n          successfulFilePath,\n          stageNumber,\n          partitionNumber,\n          workerNo\n      );\n    }\n\n    String successfulTaskId;\n\n    try (InputStream is = storageConnector.read(successfulFilePath)) {\n      successfulTaskId = IOUtils.toString(is, StandardCharsets.UTF_8);\n    }\n    if (successfulTaskId == null) {\n      throw new ISE(\"Unable to read the task id from the file: [%s]\", successfulFilePath);\n    }\n    LOG.debug(\n        \"Reading output of stage [%d], partition [%d] from task id [%s]\",","sourceCodeStart":154,"sourceCodeEnd":190,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/multi-stage-query/src/main/java/org/apache/druid/msq/shuffle/input/DurableStorageInputChannelFactory.java#L154-L190","documentation":"findSuccessfulPartitionOutput throws this ISE when the worker's '_success' marker file (got from getWorkerOutputSuccessFilePath) is absent in deep storage. The success file records which task produced the partition output; without it the consumer cannot resolve the output. Effectively the producer never signaled successful completion for this stage/partition.","triggerScenarios":"remotePartitionPath -> findSuccessfulPartitionOutput runs pathExists on the success file and it is missing: producer task crashed before writing the marker, outputs cleaned up by retention, or wrong storage prefix configured so the marker is looked up in the wrong location.","commonSituations":"Query failure after a worker was killed/OOM-killed mid-stage; durable storage cleanup interval expiring while query still running; a bug/misconfig where overlord and task use different druid.storage prefixes.","solutions":["Check the producing worker task status/logs to see if it completed and wrote the success file.","Confirm the durable-storage location and prefix configs match across services so the same path is read and written.","Retry the query; if outputs were cleaned up, reduce cleanup interval or finish queries before cleanup.","If worker crashes recur, investigate OOM/kill issues on the task runner."],"exampleFix":"// before: assuming success file exists\nString successfulFilePath = getWorkerOutputSuccessFilePath(controllerTaskId, stageNumber, workerNo);\nInputStream is = storageConnector.read(successfulFilePath);\n\n// after: validate presence and fail with actionable context\nif (!storageConnector.pathExists(successfulFilePath)) {\n  throw new ISE(\"Success marker missing; did worker task %s fail? path=%s\", workerNo, successfulFilePath);\n}","handlingStrategy":"validation","validationCode":"if (!storageConnector.pathExists(successMarkerPath)) {\n  throw new IllegalStateException(\"Producer success marker absent — producer task likely failed: \" + successMarkerPath);\n}","typeGuard":null,"tryCatchPattern":"try {\n  fetchResults();\n} catch (IllegalStateException e) {\n  if (e.getMessage().contains(\"No file present at the location\")) {\n    checkOverlordTaskStatus(controllerTaskId); // surface producer failure to user\n  }\n  throw e;\n}","preventionTips":["Check the producing task's status in the overlord before troubleshooting deep storage.","Set durable-storage cleanup intervals longer than your longest query.","Ensure consistent druid.storage config on all services."],"tags":["deep-storage","msq","success-marker","missing-file"],"backgroundTag":"file-not-found","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}