What causes "Could not resolve handle" in apache/hadoop?

Confirm the file still exists at handle.getPath() and that you are talking to the same cluster/namespace the handle was built from.

apache/hadoop · error · InvalidPathHandleException

Could not resolve handle

Error message

Could not resolve handle

What it means

HdfsPathHandle.verify(stat) throws InvalidPathHandleException('Could not resolve handle') when the resolved status is null, i.e. the path encoded in the handle no longer resolves to any file on the NameNode. PathHandles are HDFS's content/identity-addressed references (like file descriptors surviving rename); this is the 'file gone' case.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsPathHandle.java:70

  public HdfsPathHandle(ByteBuffer bytes) throws IOException {
    if (null == bytes) {
      throw new IOException("Missing PathHandle");
    }
    HdfsPathHandleProto p =
        HdfsPathHandleProto.parseFrom(ByteString.copyFrom(bytes));
    path = p.getPath();
    mtime   = p.hasMtime()   ? p.getMtime()   : null;
    inodeId = p.hasInodeId() ? p.getInodeId() : null;
  }

  public String getPath() {
    return path;
  }

  public void verify(HdfsLocatedFileStatus stat)
      throws InvalidPathHandleException {
    if (null == stat) {
      throw new InvalidPathHandleException("Could not resolve handle");
    }
    if (mtime != null && mtime != stat.getModificationTime()) {
      throw new InvalidPathHandleException("Content changed");
    }
    if (inodeId != null && inodeId != stat.getFileId()) {
      throw new InvalidPathHandleException("Wrong file");
    }
  }

  @Override
  public ByteBuffer bytes() {
    HdfsPathHandleProto.Builder b = HdfsPathHandleProto.newBuilder();
    b.setPath(path);
    if (inodeId != null) {
      b.setInodeId(inodeId);
    }
    if (mtime != null) {
      b.setMtime(mtime);

View on GitHub (pinned to 2add963021)

Solutions

Confirm the file still exists at handle.getPath() and that you are talking to the same cluster/namespace the handle was built from.
Recreate the handle from a fresh FileSystem instance if the file was re-created.
Treat this as unrecoverable for that handle: catch InvalidPathHandleException and regenerate upstream data.

Defensive patterns

Strategy: try-catch

Validate before calling

// Cheap pre-check that the handle's path still resolves before using the handle
Path p = new Path(handle.getPath());
if (!fs.exists(p)) {
  // handle cannot resolve — regenerate data instead of calling open(handle)
}

Try / catch

try (FSDataInputStream in = fs.open(handle)) {
  ...
} catch (InvalidPathHandleException e) {
  // 'Could not resolve handle': file gone or wrong cluster — unrecoverable for this handle
}

Prevention

Bind handles to a stable cluster URI before persisting them.
Do not delete files that consumers hold handles to; write new generations to new paths.
Persist enough metadata (cluster, path, creation id) beside the handle to detect mismatch early.

When it happens

Trigger: Opening data via a PathHandle (FileSystem.open(PathHandle)) after the file was deleted, moved off cluster, or the handle was built from a different namespace (wrong cluster/URI).

Common situations: Reusing serialized PathHandles across cluster migrations or restores from backup; two jobs sharing a handle where one deletes the output; typos matching handle cluster to fs.defaultFS.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/ce8cea84101daf70. Report an issue: GitHub.