apache/hadoop · error · FileNotFoundException

No such file or directory '{}'

Error message

No such file or directory '{}'

What it means

CosNFileSystem.getFileStatus throws FileNotFoundException('No such file or directory <path>') when neither an object at the exact key nor any listing entry under the key exists. CosN probes the object key and then lists one entry to detect an implicit directory; both empty means the path does not exist in the bucket. Because open, delete, rename source checks, mkdirs and create all route through getFileStatus, this error surfaces from many call sites.

Source

Thrown at hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/CosNFileSystem.java:393

    if (!key.endsWith(PATH_DELIMITER)) {
      key += PATH_DELIMITER;
    }

    // Considering that the object store's directory is a common prefix in
    // the object key, it needs to check the existence of the path by listing
    // the COS key.
    LOG.debug("List COS key: [{}] to check the existence of the path.", key);
    PartialListing listing = store.list(key, 1);
    if (listing.getFiles().length > 0
        || listing.getCommonPrefixes().length > 0) {
      if (LOG.isDebugEnabled()) {
        LOG.debug("Path: [{}] is a directory. COS key: [{}]", f, key);
      }
      return newDirectory(absolutePath);
    }

    throw new FileNotFoundException(
        "No such file or directory '" + absolutePath + "'");
  }

  @Override
  public URI getUri() {
    return uri;
  }

  /**
   * <p>
   * If <code>f</code> is a file, this method will make a single call to COS.
   * If <code>f</code> is a directory,
   * this method will make a maximum of ( <i>n</i> / 199) + 2 calls to cos,
   * where <i>n</i> is the total number of files
   * and directories contained directly in <code>f</code>.
   * </p>
   */
  @Override

View on GitHub (pinned to 2add963021)

Solutions

1. Verify with fs.exists(p) and fix the path spelling/prefix, or wait for the producer to commit.
2. Print fs.getUri() and the bucket configuration once at startup to confirm you are querying the bucket you expect.
3. If racing a producer, poll fs.exists(p) with bounded backoff instead of failing immediately.
4. List the parent directory (fs.listStatus(parent)) to see what is actually stored at that level.

Example fix

// before
FSDataInputStream in = fs.open(new Path('cosn://bkt/out/part-00000')); // FileNotFoundException

// after
Path p = new Path('cosn://bkt/out/part-00000');
if (!fs.exists(p)) {
  throw new NoSuchElementException('Input not produced: ' + p);
}
FSDataInputStream in = fs.open(p);

// Cheap pre-check; note exists() and open() still race, so pair with try-catch
if (!fs.exists(p)) {
  // handle missing input before calling open/delete/rename
}

static boolean isMissingPath(Throwable t) {
  return t instanceof FileNotFoundException;
}

try {
  return fs.open(p);
} catch (FileNotFoundException e) {
  if (awaitProducer && pollExists(fs, p, 30)) { return fs.open(p); }
  throw new NoSuchElementException('Input not produced: ' + p);
}

When it happens

Trigger: fs.open / fs.delete / fs.getFileStatus / rename with a missing source path; reading output of a job that never wrote it; wrong bucket or wrong fs.defaultFS so lookups hit a different bucket; path built with wrong prefix or separator; COS visibility lag right after an external writer uploaded the object.

Common situations: Misconfigured fs.cosn.<bucket>.bucket or credentials pointing at the wrong bucket; consuming a path before the producer committed; typo'd or case-mismatched path; assuming a directory exists when only implicit (marker-less) directories do.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• '{}' is a directory
• No such file or directory: {}
• key + ": No such file or directory."
• {} already exists
• Can not delete root path

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/0debcb5507b5f73e. Report an issue: GitHub.