apache/flink · error · IOException

file is not an instance of DistributedFileStatus

Error message

file is not an instance of DistributedFileStatus

What it means

HadoopFileSystem.getFileBlockLocations requires the passed FileStatus to be a HadoopFileStatus (or its subclass LocatedHadoopFileStatus), because it must reach through to the wrapped org.apache.hadoop.fs.FileStatus to fetch block info. Passing any other FileStatus implementation is rejected with this IOException.

Solutions

1. Obtain the FileStatus from the SAME HadoopFileSystem instance: use hadoopFs.getFileStatus(path) and pass that result to getFileBlockLocations
2. Prefer the path-based overload getFileBlockLocations(path, start, len) which avoids the type coupling entirely

Example fix

// before
FileStatus st = otherFs.getFileStatus(path);
hadoopFs.getFileBlockLocations(st, 0, len); // throws
// after
FileStatus st = hadoopFs.getFileStatus(path);
hadoopFs.getFileBlockLocations(st, 0, len);

// ensure status came from this filesystem before querying locations
FileStatus status = fs.getFileStatus(path); // same fs instance
BlockLocation[] locs = fs.getFileBlockLocations(status, 0, status.getLen());

boolean isHadoopStatus(FileStatus s) {
    return s instanceof org.apache.flink.runtime.fs.hdfs.HadoopFileSystem.HadoopFileStatus;
}

try {
    locs = fs.getFileBlockLocations(status, 0, len);
} catch (IOException e) {
    if (e.getMessage().contains("not an instance of")) {
        // re-fetch status from this fs, or use the path-based overload
    }
}

When it happens

Trigger: Calling getFileBlockLocations(fileStatus, start, len) with a FileStatus obtained from a different FileSystem implementation (e.g. local or another plugin), a hand-constructed FileStatus, or a copied/deserialized status object.

Common situations: Code that mixes filesystems: stat a path on one filesystem, then ask another filesystem for block locations; generic utility code that accepts FileSystem + FileStatus from different sources; caching FileStatus objects across filesystem instances.

Related errors

• Hostname starts with a
• The given offset is not contained in the any block.
• A predicate on a BOOLEAN column requires a Boolean literal.
• A predicate on a FLOAT column requires a floating literal…
• A predicate on a LONG column requires an integer literal…

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/ff57aa3d1af30a80. Report an issue: GitHub.

Appendix: source

Thrown at flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopFileSystem.java:96

        return new Path(this.fs.getHomeDirectory().toUri());
    }

    @Override
    public URI getUri() {
        return fs.getUri();
    }

    @Override
    public FileStatus getFileStatus(final Path f) throws IOException {
        org.apache.hadoop.fs.FileStatus status = this.fs.getFileStatus(toHadoopPath(f));
        return HadoopFileStatus.fromHadoopStatus(status);
    }

    @Override
    public BlockLocation[] getFileBlockLocations(
            final FileStatus file, final long start, final long len) throws IOException {
        if (!(file instanceof HadoopFileStatus)) {
            throw new IOException("file is not an instance of DistributedFileStatus");
        }

        // shortcut - if the status already has the information, return it.
        if (file instanceof LocatedHadoopFileStatus) {
            return ((LocatedHadoopFileStatus) file).getBlockLocations();
        }

        final org.apache.hadoop.fs.FileStatus hadoopStatus =
                ((HadoopFileStatus) file).getInternalFileStatus();

        // second shortcut - if the internal status already has the information, return it.
        // only if that is not the case, to the actual HDFS call (RPC to Name Node)
        final org.apache.hadoop.fs.BlockLocation[] blkLocations =
                hadoopStatus instanceof org.apache.hadoop.fs.LocatedFileStatus
                        ? ((org.apache.hadoop.fs.LocatedFileStatus) hadoopStatus)
                                .getBlockLocations()
                        : fs.getFileBlockLocations(hadoopStatus, start, len);

View on GitHub (pinned to 2f3c205e92)