{"record":{"id":"c4b047b58d6903af","repo":"apache/hadoop","slug":"no-such-file-or-directory-path","errorCode":null,"errorMessage":"No such file or directory: <path>","messagePattern":"No such file or directory: <path>","errorType":"exception","errorClass":"FileNotFoundException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java","lineNumber":4118,"sourceCode":"          }\n          // either an empty directory is not needed, or the\n          // listing does not meet the requirements.\n          return new S3AFileStatus(Tristate.FALSE, path, username);\n        } else if (key.isEmpty()) {\n          LOG.debug(\"Found root directory\");\n          return new S3AFileStatus(Tristate.TRUE, path, username);\n        }\n      } catch (AwsServiceException e) {\n        if (e.statusCode() != SC_404_NOT_FOUND || isUnknownBucket(e)) {\n          throw translateException(\"getFileStatus\", path, e);\n        }\n      } catch (SdkException e) {\n        throw translateException(\"getFileStatus\", path, e);\n      }\n    }\n\n    LOG.debug(\"Not Found: {}\", path);\n    throw new FileNotFoundException(\"No such file or directory: \" + path);\n  }\n\n  /**\n   * Probe S3 for a file or dir existing, with the given probe set.\n   * Retry policy: retrying; translated.\n   * @param path qualified path to look for\n   * @param probes probes to make\n   * @return true if path exists in S3\n   * @throws IOException IO failure\n   */\n  @Retries.RetryTranslated\n  private boolean s3Exists(final Path path, final Set<StatusProbeEnum> probes)\n      throws IOException {\n    String key = pathToKey(path);\n    try {\n      s3GetFileStatus(path, key, probes, false);\n      return true;\n    } catch (FileNotFoundException e) {","sourceCodeStart":4100,"sourceCodeEnd":4136,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L4100-L4136","documentation":"innerGetFileStatus() exhausts its probes (HEAD, and LIST per the probe set) without finding the path: the AwsServiceException was a 404 that is not an unknown-bucket signal (statusCode() == 404 && !isUnknownBucket(e)) - anything else is translated via translateException(). It then throws FileNotFoundException('No such file or directory: <path>'): the store was queried successfully and nothing is there.","triggerScenarios":"open/getFileStatus/listStatus on a path with no object and no directory marker; reading job output before it is committed/published; the object deleted by another client between an earlier listing and this call.","commonSituations":"Consuming upstream output too early; path casing or construction mistakes (S3 keys are case-sensitive); files removed by lifecycle rules or concurrent cleanup.","solutions":["Verify the path against the store directly (aws s3 ls with the same bucket/endpoint/credentials)","Handle absence as a workflow event: fs.exists() probe or catch FileNotFoundException and wait/recompute as appropriate","Fix path construction (casing, qualification, missing elements)"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Optional-based existence check before opening\nif (!fs.exists(path)) {\n  return Optional.empty();\n}\ntry (FSDataInputStream in = fs.open(path)) { ... }","typeGuard":"static boolean isMissing(Throwable t) {\n  return t instanceof java.io.FileNotFoundException;\n}","tryCatchPattern":"Catch FileNotFoundException separately from IOException around open/getFileStatus; treat it as 'absent on the store' - skip, wait for upstream publication, or fix the path. Never retry blindly: the object will not appear by retrying the same call.","preventionTips":["Check existence (fs.exists / getFileStatus) before opening paths produced by other jobs","Normalize and case-check path construction - S3 keys are case-sensitive","In pipelines, consume only committed/published output paths"],"tags":["s3a","file-not-found","status","probe"],"backgroundTag":"file-not-found","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}