{"record":{"id":"193b49938ce5ca80","repo":"apache/flink","slug":"file-already-exists-193b49","errorCode":null,"errorMessage":"File already exists: {}","messagePattern":"File already exists: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/NativeS3FileSystem.java","lineNumber":438,"sourceCode":"     *\n     * <p>If explicit directory markers are needed, consider using a custom implementation.\n     *\n     * @return always returns true (S3 doesn't require explicit directory creation)\n     */\n    @Override\n    public boolean mkdirs(Path path) throws IOException {\n        checkNotClosed();\n        LOG.debug(\"mkdirs called for {} - S3 doesn't require explicit directory creation\", path);\n        return true;\n    }\n\n    @Override\n    public FSDataOutputStream create(Path path, WriteMode overwriteMode) throws IOException {\n        checkNotClosed();\n        if (overwriteMode == WriteMode.NO_OVERWRITE) {\n            try {\n                if (exists(path)) {\n                    throw new IOException(\"File already exists: \" + path);\n                }\n            } catch (FileNotFoundException ignored) {\n            }\n        } else {\n            try {\n                delete(path, false);\n            } catch (FileNotFoundException ignored) {\n            }\n        }\n\n        final String key = NativeS3ObjectOperations.extractKey(path);\n        return new NativeS3OutputStream(\n                clientProvider.getS3Client(),\n                bucketName,\n                key,\n                localTmpDir,\n                clientProvider.getEncryptionConfig());\n    }","sourceCodeStart":420,"sourceCodeEnd":456,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/NativeS3FileSystem.java#L420-L456","documentation":"NativeS3FileSystem.create with WriteMode.NO_OVERWRITE first checks exists(path) and throws IOException('File already exists: <path>') if an object already exists at that key. This implements the no-clobber contract required by Flink's FileSystem API and output committers that must not overwrite committed results.","triggerScenarios":"Calling create(path, WriteMode.NO_OVERWRITE) (or an API defaulting to no-overwrite) when the S3 key already exists — e.g. re-running a job without cleanup, task retry writing the same part file, or output directory reuse.","commonSituations":"Re-submitting a failed job against the same output prefix without deleting previous part files; concurrent tasks computing identical output filenames; recovery replaying a write after the object was already committed.","solutions":["Delete or move the existing object/directory before re-running: s3Fs.delete(path, true).","Use WriteMode.OVERWRITE when clobbering is acceptable (create then internally deletes the existing object).","Make output filenames unique per attempt (e.g. include subtask index and attempt number) so retries never collide."],"exampleFix":"// before\nFSDataOutputStream out = s3Fs.create(path, WriteMode.NO_OVERWRITE);\n\n// after\nFSDataOutputStream out = s3Fs.create(path, WriteMode.OVERWRITE);\n// or before creating:\nif (s3Fs.exists(path)) {\n    s3Fs.delete(path, false);\n}","handlingStrategy":"validation","validationCode":"// pre-check when NO_OVERWRITE semantics matter\nif (s3Fs.exists(path)) {\n    throw new IOException(\"Output already exists: \" + path + \" — clean up or use unique names\");\n}\nFSDataOutputStream out = s3Fs.create(path, WriteMode.NO_OVERWRITE);","typeGuard":null,"tryCatchPattern":"try {\n    out = s3Fs.create(path, WriteMode.NO_OVERWRITE);\n} catch (IOException e) {\n    if (e.getMessage() != null && e.getMessage().startsWith(\"File already exists\")) {\n        // collision: choose a unique name (add attempt id) or delete old output first\n    } else {\n        throw e;\n    }\n}","preventionTips":["Include subtask index and attempt number in output filenames so retries never collide.","Clean output prefixes before re-running jobs that use NO_OVERWRITE.","Treat this exception as a signal of duplicate output, not an S3 fault."],"tags":["s3","create","overwrite-mode","idempotency"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}