apache/hadoop · error · FileNotFoundException

No such file or directory: <path>

Error message

No such file or directory: <path>

What it means

innerGetFileStatus() exhausts its probes (HEAD, and LIST per the probe set) without finding the path: the AwsServiceException was a 404 that is not an unknown-bucket signal (statusCode() == 404 && !isUnknownBucket(e)) - anything else is translated via translateException(). It then throws FileNotFoundException('No such file or directory: <path>'): the store was queried successfully and nothing is there.

Source

Thrown at hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java:4118

          }
          // either an empty directory is not needed, or the
          // listing does not meet the requirements.
          return new S3AFileStatus(Tristate.FALSE, path, username);
        } else if (key.isEmpty()) {
          LOG.debug("Found root directory");
          return new S3AFileStatus(Tristate.TRUE, path, username);
        }
      } catch (AwsServiceException e) {
        if (e.statusCode() != SC_404_NOT_FOUND || isUnknownBucket(e)) {
          throw translateException("getFileStatus", path, e);
        }
      } catch (SdkException e) {
        throw translateException("getFileStatus", path, e);
      }
    }

    LOG.debug("Not Found: {}", path);
    throw new FileNotFoundException("No such file or directory: " + path);
  }

  /**
   * Probe S3 for a file or dir existing, with the given probe set.
   * Retry policy: retrying; translated.
   * @param path qualified path to look for
   * @param probes probes to make
   * @return true if path exists in S3
   * @throws IOException IO failure
   */
  @Retries.RetryTranslated
  private boolean s3Exists(final Path path, final Set<StatusProbeEnum> probes)
      throws IOException {
    String key = pathToKey(path);
    try {
      s3GetFileStatus(path, key, probes, false);
      return true;
    } catch (FileNotFoundException e) {

View on GitHub (pinned to 2add963021)

Solutions

1. Verify the path against the store directly (aws s3 ls with the same bucket/endpoint/credentials)
2. Handle absence as a workflow event: fs.exists() probe or catch FileNotFoundException and wait/recompute as appropriate
3. Fix path construction (casing, qualification, missing elements)

// Optional-based existence check before opening
if (!fs.exists(path)) {
  return Optional.empty();
}
try (FSDataInputStream in = fs.open(path)) { ... }

static boolean isMissing(Throwable t) {
  return t instanceof java.io.FileNotFoundException;
}

Catch FileNotFoundException separately from IOException around open/getFileStatus; treat it as 'absent on the store' - skip, wait for upstream publication, or fix the path. Never retry blindly: the object will not appear by retrying the same call.

When it happens

Trigger: open/getFileStatus/listStatus on a path with no object and no directory marker; reading job output before it is committed/published; the object deleted by another client between an earlier listing and this call.

Common situations: Consuming upstream output too early; path casing or construction mistakes (S3 keys are case-sensitive); files removed by lifecycle rules or concurrent cleanup.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• key + ": No such file or directory."
• No such file or directory '{}'
• '{}' is a directory
• Can not create '%s' file, because parent folder does not exi
• %s not found: %s

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/c4b047b58d6903af. Report an issue: GitHub.