apache/flink · error · FileNotFoundException

File not found: {}

Error message

File not found: {}

What it means

NativeS3FileSystem.getFileStatus, when the key is not an object, calls getDirectoryStatus which lists the bucket with the key as prefix (maxKeys=1). If no objects and no common prefixes exist under that prefix, the path is neither a file nor a non-empty directory, so FileNotFoundException('File not found: <path>') is thrown, matching standard FileSystem contract.

Source

Thrown at flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/NativeS3FileSystem.java:281

            throw S3ExceptionUtils.toIOException(
                    String.format("Failed to get file status for s3://%s/%s", bucketName, key), e);
        }
    }

    /**
     * Checks if the given key represents a directory by listing objects with that prefix. Returns a
     * directory {@link FileStatus} if objects exist under the prefix, otherwise throws {@link
     * FileNotFoundException}.
     */
    private FileStatus getDirectoryStatus(S3Client s3Client, String key, Path path)
            throws FileNotFoundException {
        final String prefix = key.endsWith("/") ? key : key + "/";
        final ListObjectsV2Request listRequest =
                ListObjectsV2Request.builder().bucket(bucketName).prefix(prefix).maxKeys(1).build();
        final ListObjectsV2Response listResponse = s3Client.listObjectsV2(listRequest);

        if (listResponse.contents().isEmpty() && !listResponse.hasCommonPrefixes()) {
            throw new FileNotFoundException("File not found: " + path);
        }

        LOG.debug("Path is a directory: {}", key);
        return S3FileStatus.withDirectory(path);
    }

    @Override
    public BlockLocation[] getFileBlockLocations(FileStatus file, long start, long len) {
        return new BlockLocation[] {
            new S3BlockLocation(new String[] {"localhost"}, 0, file.getLen())
        };
    }

    @Override
    public FSDataInputStream open(Path path, int bufferSize) throws IOException {
        checkNotClosed();
        final String key = NativeS3ObjectOperations.extractKey(path);
        final S3Client s3Client = clientProvider.getS3Client();

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Verify the exact key spelling, bucket, and scheme in the path against the S3 console/listing.
2. Use exists() or catch FileNotFoundException to handle absent paths gracefully instead of failing the job.
3. If the path is a directory, ensure at least one object exists under its prefix — empty directories are not represented.

Example fix

// before
FileStatus st = s3Fs.getFileStatus(new Path("s3://bucket/typo/path"));

// after
Path p = new Path("s3://bucket/typo/path");
if (!s3Fs.exists(p)) {
    throw new FileNotFoundException("Path does not exist, check bucket/key: " + p);
}
FileStatus st = s3Fs.getFileStatus(p);

// cheap pre-check
if (!s3Fs.exists(path)) {
    // handle absent path before calling getFileStatus/open
}

try {
    return s3Fs.getFileStatus(path);
} catch (FileNotFoundException e) {
    // expected for absent keys on S3: treat as empty/absent input, do not fail the job
    return null;
}

When it happens

Trigger: Calling getFileStatus, exists, open, or delete on an S3 path whose key and key-as-prefix both match nothing in the bucket — a typo'd path, an object that was deleted, or a directory that never existed (S3 has no explicit directory markers here).

Common situations: Reading from a wrong bucket/path; source objects not yet uploaded when the job starts; paths built with double slashes or missing prefixes; assuming mkdirs creates a visible empty directory (S3 here only recognizes directories that contain objects).

Related errors

• Directory not empty and recursive = false
• NativeS3FileSystem does not support renaming directories: {}
• Input opening request timed out. Opener was {} alive. Stack
• Output path could not be initialized.
• Cannot sync state to system like S3. Use persist() to create

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/f95ccb440add0320. Report an issue: GitHub.