{"record":{"id":"cfbdc82c43d71204","repo":"apache/flink","slug":"cannot-create-local-tmp-dir","errorCode":null,"errorMessage":"Cannot create local tmp dir: {}","messagePattern":"Cannot create local tmp dir: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/writer/NativeS3RecoverableWriter.java","lineNumber":150,"sourceCode":"                } catch (IOException cleanup) {\n                    e.addSuppressed(cleanup);\n                }\n            }\n            throw e;\n        }\n    }\n\n    /**\n     * Downloads the side object holding the previously-persisted sub-part-size tail into a fresh\n     * file under {@link #localTmpDir}. The side object itself is left in place so that a repeated\n     * recovery from the same checkpoint remains correct; cleanup is the responsibility of {@link\n     * #cleanupRecoverableState(ResumeRecoverable)} which Flink invokes when the checkpoint is\n     * retired.\n     */\n    private File downloadIncompleteTail(NativeS3Recoverable s3recoverable) throws IOException {\n        final File tmpDir = new File(localTmpDir);\n        if (!tmpDir.exists() && !tmpDir.mkdirs()) {\n            throw new IOException(\"Cannot create local tmp dir: \" + localTmpDir);\n        }\n        final File target = new File(tmpDir, \"s3-resume-\" + UUID.randomUUID());\n        try {\n            final long downloaded =\n                    s3AccessHelper.getObject(s3recoverable.incompleteObjectName(), target);\n            if (downloaded != s3recoverable.incompleteObjectLength()) {\n                throw new IOException(\n                        \"Incomplete-tail object \"\n                                + s3recoverable.incompleteObjectName()\n                                + \" has unexpected length (expected \"\n                                + s3recoverable.incompleteObjectLength()\n                                + \" bytes, got \"\n                                + downloaded\n                                + \" bytes). The side object holding the in-flight tail \"\n                                + \"has been truncated, overwritten, or replaced out-of-band \"\n                                + \"since the checkpoint was taken. Recovery cannot proceed: \"\n                                + \"the writer state is inconsistent with the checkpoint and \"\n                                + \"this failure is NOT retriable from the same checkpoint. \"","sourceCodeStart":132,"sourceCodeEnd":168,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/writer/NativeS3RecoverableWriter.java#L132-L168","documentation":"Thrown by NativeS3RecoverableWriter.downloadIncompleteTail when the local tmp directory neither exists nor can be created with mkdirs(). Recovery needs scratch space to download the in-flight tail object; without it, resume cannot proceed.","triggerScenarios":"recover() with a non-null incomplete object when localTmpDir is unwritable (permissions), its parent is a file, the disk is full, or the path is invalid on the filesystem (e.g. embedded NUL, or a read-only mount). mkdirs() returning false covers all non-exception failure modes.","commonSituations":"TaskManager running as a user without write permission to the configured tmp dir; container with read-only root FS and tmp dir pointing inside it; disk exhaustion; localTmpDir defaulting into a path that another job holds as a regular file; SELinux/AppArmor denying writes.","solutions":["Check the configured local temporary directory (S3 filesystem config) and fix ownership/permissions so the Flink process can create it.","Point the tmp dir at writable, adequately-sized local storage (not read-only mounts); prefer a dedicated volume on TaskManagers.","Free disk space or enlarge the volume if full — multipart tails plus part buffering need headroom.","Verify with a manual test on the node: sudo -u <flink-user> mkdir -p <localTmpDir>."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"File d = new File(localTmpDir);\nif (!d.exists() && !d.mkdirs()) {\n    throw new IllegalStateException(\"tmp dir not creatable: \" + localTmpDir);\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Pre-create and chown the tmp dir in container images.","Monitor local disk space on TaskManagers.","Point tmp dir at writable scratch volumes, not read-only mounts."],"tags":["s3","local-disk","permissions","recovery","config"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}