apache/hadoop · error · IOException

Cannot open filename {}

Error message

Cannot open filename {}

What it means

DFSClient.open() first fetches LocatedBlocks from the NameNode; a null response means no file exists at that src, and openInternal throws a plain IOException('Cannot open filename ' + src) rather than FileNotFoundException. This typing wart means a missing file at open time masquerades as a generic IO error unless you re-check existence yourself.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java:1136

    try (TraceScope ignored = newPathTraceScope("newDFSInputStream", src)) {
      HdfsLocatedFileStatus s = getLocatedFileInfo(src, true);
      fd.verify(s); // check invariants in path handle
      LocatedBlocks locatedBlocks = s.getLocatedBlocks();
      return openInternal(locatedBlocks, src, verifyChecksum);
    }
  }

  private DFSInputStream openInternal(LocatedBlocks locatedBlocks, String src,
      boolean verifyChecksum) throws IOException {
    if (locatedBlocks != null) {
      ErasureCodingPolicy ecPolicy = locatedBlocks.getErasureCodingPolicy();
      if (ecPolicy != null) {
        return new DFSStripedInputStream(this, src, verifyChecksum, ecPolicy,
            locatedBlocks);
      }
      return new DFSInputStream(this, src, verifyChecksum, locatedBlocks);
    } else {
      throw new IOException("Cannot open filename " + src);
    }
  }

  /**
   * Get the namenode associated with this DFSClient object
   * @return the namenode associated with this DFSClient object
   */
  public ClientProtocol getNamenode() {
    return namenode;
  }

  /**
   * Call {@link #create(String, boolean, short, long, Progressable)} with
   * default <code>replication</code> and <code>blockSize</code> and null
   * <code>progress</code>.
   */
  public OutputStream create(String src, boolean overwrite)
      throws IOException {

View on GitHub (pinned to 2add963021)

Solutions

1. Pre-check existence immediately before open and throw a precise FileNotFoundException yourself when absent.
2. In the catch for open(), disambiguate by re-checking existence: the IOException subtype will not tell you it is a missing file.
3. For read-after-write races, have the producer publish via atomic rename of a temp file and retry open with backoff until it appears.
4. Log the exact src string handed to open to catch path-assembly bugs.

Example fix

// before
FSDataInputStream in = dfsClient.open(src);
// IOException: Cannot open filename /path/file

// after
if (dfsClient.getFileInfo(src) == null) {
  throw new FileNotFoundException('Cannot open ' + src + ': not in namespace');
}
FSDataInputStream in = dfsClient.open(src);

if (!fs.exists(path)) {
  throw new FileNotFoundException(path + " missing before open");
}

catch (IOException e) {
  // open() throws generic IOException for missing files: re-check to classify
  if (!fs.exists(path)) { /* missing file: recreate/skip/fail with context */ }
  else { throw e; }
}

When it happens

Trigger: dfsClient.open(src) when src never existed or was deleted between an exists()/getFileStatus() check and the open; readers racing a writer's create or a cleanup's delete; paths assembled from wrong variables.

Common situations: Test pipelines reading output of a job that failed before creating the file; consumers opening files after upstream retention cleanup; create-then-open races in integration tests without synchronization.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• File not found: {}, likely due to delayed block removal
• Directory does not exist: {}
• File does not exist: {}
• File does not exist: {}
• %s does not exist or is not file.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a8769a714ec89dde. Report an issue: GitHub.