{"record":{"id":"fd3e8d0d4a0bbd85","repo":"apache/flink","slug":"natives3filesystem-does-not-support-renaming-direc","errorCode":null,"errorMessage":"NativeS3FileSystem does not support renaming directories: {}","messagePattern":"NativeS3FileSystem does not support renaming directories: (.+?)","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/NativeS3FileSystem.java","lineNumber":474,"sourceCode":"    }\n\n    /**\n     * Renames a single file from {@code src} to {@code dst}.\n     *\n     * <p><b>Directory rename is not supported.</b>\n     *\n     * @throws UnsupportedOperationException if {@code src} is a directory\n     */\n    @Override\n    public boolean rename(Path src, Path dst) throws IOException {\n        checkNotClosed();\n        final String srcKey = NativeS3ObjectOperations.extractKey(src);\n        final String dstKey = NativeS3ObjectOperations.extractKey(dst);\n        final S3Client s3Client = clientProvider.getS3Client();\n\n        final FileStatus srcStatus = getFileStatus(src);\n        if (srcStatus.isDir()) {\n            throw new UnsupportedOperationException(\n                    \"NativeS3FileSystem does not support renaming directories: \" + src);\n        }\n\n        try {\n            final CopyObjectRequest copyRequest =\n                    CopyObjectRequest.builder()\n                            .sourceBucket(bucketName)\n                            .sourceKey(srcKey)\n                            .destinationBucket(bucketName)\n                            .destinationKey(dstKey)\n                            .build();\n            s3Client.copyObject(copyRequest);\n            final DeleteObjectRequest deleteRequest =\n                    DeleteObjectRequest.builder().bucket(bucketName).key(srcKey).build();\n            s3Client.deleteObject(deleteRequest);\n            return true;\n        } catch (S3Exception e) {\n            throw new IOException(\"Failed to rename \" + src + \" to \" + dst, e);","sourceCodeStart":456,"sourceCodeEnd":492,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/NativeS3FileSystem.java#L456-L492","documentation":"NativeS3FileSystem.rename only supports single-object renames (copy + delete). If getFileStatus(src).isDir(), it throws UnsupportedOperationException, because renaming a directory prefix in S3 requires copying every object under the prefix, which this implementation deliberately does not do (cost/consistency reasons).","triggerScenarios":"Calling rename() where src resolves to a directory (objects exist under src/ prefix or a directory marker exists) — e.g. commit-time directory moves, moving date-partitioned output trees, or filesystem tests that assume POSIX rename.","commonSituations":"Porting HDFS-style jobs that atomically move output directories on commit; using S3 with frameworks that rely on directory rename for transactions (the classic S3A consistency limitation); moving partition folders after writing.","solutions":["Rename objects individually: list all keys under the prefix, copyObject each to the new prefix, then delete the sources (wrap in a loop with your own error handling).","Write directly to the final destination keys instead of moving directories afterwards.","Use a committer designed for S3 (e.g. S3 committers via magic/staged commits) instead of rename-based commit protocols."],"exampleFix":"// before\nboolean ok = s3Fs.rename(srcDir, dstDir); // UnsupportedOperationException\n\n// after\n// rename objects one by one under the prefix\nfor (FileStatus st : s3Fs.listStatus(srcDir)) {\n    Path dst = new Path(dstDir, st.getPath().getName());\n    if (!s3Fs.rename(st.getPath(), dst)) {\n        throw new IOException(\"Failed to move \" + st.getPath());\n    }\n}","handlingStrategy":"type-guard","validationCode":"// guard: only rename files, never directories\nFileStatus st = s3Fs.getFileStatus(src);\nif (st.isDir()) {\n    throw new UnsupportedOperationException(\"Directory rename not supported; move children individually\");\n}","typeGuard":"static boolean isFileRenameSupported(FileSystem fs, Path src) throws IOException {\n    return !fs.getFileStatus(src).isDir(); // NativeS3FileSystem renames only objects\n}","tryCatchPattern":"try {\n    s3Fs.rename(src, dst);\n} catch (UnsupportedOperationException e) {\n    // directory rename: fall back to per-object copy+delete under the prefix\n    for (FileStatus child : s3Fs.listStatus(src)) {\n        s3Fs.rename(child.getPath(), new Path(dst, child.getPath().getName()));\n    }\n}","preventionTips":["Never rely on directory rename for commit protocols on S3; use S3-aware committers.","Write data directly to final keys rather than staging and moving directories.","Guard every rename call with an isDir() check."],"tags":["s3","rename","directory","object-store-semantics","unsupported-operation"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}