{"record":{"id":"0ad95a1f419c8041","repo":"apache/flink","slug":"missing-data-in-tmp-file","errorCode":null,"errorMessage":"Missing data in tmp file: {}","messagePattern":"Missing data in tmp file: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"flink-core/src/main/java/org/apache/flink/core/fs/local/LocalRecoverableFsDataOutputStream.java","lineNumber":77,"sourceCode":"        this.fileChannel =\n                FileChannel.open(\n                        tempFile.toPath(), StandardOpenOption.WRITE, StandardOpenOption.CREATE_NEW);\n        this.fos = Channels.newOutputStream(fileChannel);\n    }\n\n    LocalRecoverableFsDataOutputStream(LocalRecoverable resumable) throws IOException {\n        this.targetFile = checkNotNull(resumable.targetFile());\n        this.tempFile = checkNotNull(resumable.tempFile());\n\n        if (!tempFile.exists()) {\n            throw new FileNotFoundException(\"File Not Found: \" + tempFile.getAbsolutePath());\n        }\n\n        this.fileChannel =\n                FileChannel.open(\n                        tempFile.toPath(), StandardOpenOption.WRITE, StandardOpenOption.APPEND);\n        if (this.fileChannel.position() < resumable.offset()) {\n            throw new IOException(\"Missing data in tmp file: \" + tempFile.getAbsolutePath());\n        }\n        this.fileChannel.truncate(resumable.offset());\n        this.fos = Channels.newOutputStream(fileChannel);\n    }\n\n    @VisibleForTesting\n    LocalRecoverableFsDataOutputStream(\n            File targetFile, File tempFile, FileChannel fileChannel, OutputStream fos) {\n        this.targetFile = checkNotNull(targetFile);\n        this.tempFile = checkNotNull(tempFile);\n        this.fileChannel = fileChannel;\n        this.fos = fos;\n    }\n\n    @Override\n    public void write(int b) throws IOException {\n        fos.write(b);\n    }","sourceCodeStart":59,"sourceCodeEnd":95,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-core/src/main/java/org/apache/flink/core/fs/local/LocalRecoverableFsDataOutputStream.java#L59-L95","documentation":"Thrown by the LocalRecoverableFsDataOutputStream recovery constructor when the opened temp file's channel position is less than the expected offset recorded in the LocalRecoverable. This means bytes that were supposedly persisted are missing from the temp file, so the recovered stream would be inconsistent.","triggerScenarios":"Constructing the recovery stream where tempFile exists but its size < resumable.offset(); i.e. the temp file was truncated or partially lost since persist() was called.","commonSituations":"External truncation of the temp file; disk space reclamation; partial write that lost tail bytes; manual tampering with the temp file; filesystem corruption.","solutions":["Discard this recovery attempt and recompute from the last fully-committed checkpoint.","Ensure the temp directory is on stable storage that does not truncate files.","Audit for any process or cron job that may truncate/clean temp files.","If reproducible, capture the temp file and the LocalRecoverable for root-cause analysis."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"void checkRecoverableIntegrity(LocalRecoverable r) throws IOException {\n    long len = r.tempFile().length();\n    if (len < r.offset())\n        throw new IOException(\"temp file shorter than expected offset; data lost\");\n}","typeGuard":null,"tryCatchPattern":"try {\n    new LocalRecoverableFsDataOutputStream(resumable);\n} catch (IOException e) {\n    if (e.getMessage().startsWith(\"Missing data in tmp file\")) {\n        // discard and recompute from last checkpoint\n    }\n}","preventionTips":["Place temp files on storage that does not truncate.","Investigate any cleanup daemon touching the temp dir.","Treat offset mismatches as data-loss signals and fall back to checkpoint."],"tags":["filesystem","local-fs","recoverable-writer","recovery","data-loss"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}