apache/flink · error · IOException

Missing data in tmp file: {}

Error message

Missing data in tmp file: {}

What it means

Thrown by the LocalRecoverableFsDataOutputStream recovery constructor when the opened temp file's channel position is less than the expected offset recorded in the LocalRecoverable. This means bytes that were supposedly persisted are missing from the temp file, so the recovered stream would be inconsistent.

Source

Thrown at flink-core/src/main/java/org/apache/flink/core/fs/local/LocalRecoverableFsDataOutputStream.java:77

        this.fileChannel =
                FileChannel.open(
                        tempFile.toPath(), StandardOpenOption.WRITE, StandardOpenOption.CREATE_NEW);
        this.fos = Channels.newOutputStream(fileChannel);
    }

    LocalRecoverableFsDataOutputStream(LocalRecoverable resumable) throws IOException {
        this.targetFile = checkNotNull(resumable.targetFile());
        this.tempFile = checkNotNull(resumable.tempFile());

        if (!tempFile.exists()) {
            throw new FileNotFoundException("File Not Found: " + tempFile.getAbsolutePath());
        }

        this.fileChannel =
                FileChannel.open(
                        tempFile.toPath(), StandardOpenOption.WRITE, StandardOpenOption.APPEND);
        if (this.fileChannel.position() < resumable.offset()) {
            throw new IOException("Missing data in tmp file: " + tempFile.getAbsolutePath());
        }
        this.fileChannel.truncate(resumable.offset());
        this.fos = Channels.newOutputStream(fileChannel);
    }

    @VisibleForTesting
    LocalRecoverableFsDataOutputStream(
            File targetFile, File tempFile, FileChannel fileChannel, OutputStream fos) {
        this.targetFile = checkNotNull(targetFile);
        this.tempFile = checkNotNull(tempFile);
        this.fileChannel = fileChannel;
        this.fos = fos;
    }

    @Override
    public void write(int b) throws IOException {
        fos.write(b);
    }

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Discard this recovery attempt and recompute from the last fully-committed checkpoint.
2. Ensure the temp directory is on stable storage that does not truncate files.
3. Audit for any process or cron job that may truncate/clean temp files.
4. If reproducible, capture the temp file and the LocalRecoverable for root-cause analysis.

void checkRecoverableIntegrity(LocalRecoverable r) throws IOException {
    long len = r.tempFile().length();
    if (len < r.offset())
        throw new IOException("temp file shorter than expected offset; data lost");
}

try {
    new LocalRecoverableFsDataOutputStream(resumable);
} catch (IOException e) {
    if (e.getMessage().startsWith("Missing data in tmp file")) {
        // discard and recompute from last checkpoint
    }
}

When it happens

Trigger: Constructing the recovery stream where tempFile exists but its size < resumable.offset(); i.e. the temp file was truncated or partially lost since persist() was called.

Common situations: External truncation of the temp file; disk space reclamation; partial write that lost tail bytes; manual tampering with the temp file; filesystem corruption.

Related errors

• File Not Found: {}
• Cannot clean commit: File has trailing junk data.
• Committing file failed, could not rename {} -> {}
• Committing file failed. Target file already exists: {}
• Cannot sync state to system like S3. Use persist() to create

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/0ad95a1f419c8041. Report an issue: GitHub.