apache/seatunnel · error · CheckpointStorageException

Failed to get all checkpoints for job ${jobId}

Error message

Failed to get all checkpoints for job ${jobId}

What it means

LocalFileStorage.getAllCheckpoints throws CheckpointStorageException when recursively listing checkpoint data files under the job's storage directory fails. The library wraps any exception from FileUtils.listFiles (IO errors, permission problems, path issues) into this checked storage exception so callers of the checkpoint storage SPI get a uniform error type. It is thrown before any checkpoint deserialization happens, so it indicates filesystem-level failure, not corrupt checkpoint data.

Source

Thrown at seatunnel-engine/seatunnel-engine-storage/checkpoint-storage-plugins/checkpoint-storage-local-file/src/main/java/org/apache/seatunnel/engine/checkpoint/storage/localfile/LocalFileStorage.java:126

            throw new CheckpointStorageException(
                    "Failed to write checkpoint data to file " + fileName, e);
        }

        return fileName;
    }

    @Override
    public List<PipelineState> getAllCheckpoints(String jobId) throws CheckpointStorageException {
        File filePath = new File(getStorageParentDirectory() + jobId);
        if (!filePath.exists()) {
            return new ArrayList<>();
        }

        Collection<File> fileList;
        try {
            fileList = FileUtils.listFiles(filePath, FILE_EXTENSIONS, true);
        } catch (Exception e) {
            throw new CheckpointStorageException(
                    "Failed to get all checkpoints for job " + jobId, e);
        }
        if (fileList.isEmpty()) {
            log.info("No checkpoint found for this job, the job id is: " + jobId);
            return new ArrayList<>();
        }
        List<PipelineState> states = new ArrayList<>();
        fileList.forEach(
                file -> {
                    try {
                        byte[] data = FileUtils.readFileToByteArray(file);
                        states.add(deserializeCheckPointData(data));
                    } catch (IOException e) {
                        log.error(
                                "Failed to read checkpoint data from file "
                                        + file.getAbsolutePath(),
                                e);
                    }

View on GitHub (pinned to cf67b549a7)

Solutions

1. Check OS permissions on the checkpoint storage directory and ensure the process user can read/traverse it: ls -la on getStorageParentDirectory()+jobId.
2. Verify the checkpoint storage parent path configuration points at an existing, mounted directory.
3. If on NFS/network storage, confirm the mount is alive and retry the job or recovery.
4. Check the cause exception attached to the CheckpointStorageException for the exact filesystem error.

Example fix

// before
fileList = FileUtils.listFiles(filePath, FILE_EXTENSIONS, true);
// after (pre-validate the directory before listing)
if (!Files.isDirectory(filePath)) {
    log.warn("Checkpoint directory missing for job " + jobId + ", returning empty");
    return new ArrayList<>();
}
fileList = FileUtils.listFiles(filePath, FILE_EXTENSIONS, true);

// Java: verify the job checkpoint directory is readable before recovery
java.nio.file.Path dir = java.nio.file.Path.of(storageParentDir, jobId);
if (!java.nio.file.Files.isDirectory(dir) || !java.nio.file.Files.isReadable(dir)) {
    throw new IllegalStateException("Checkpoint dir missing or unreadable: " + dir);
}

static boolean isReadableDirectory(Path p) {
    return p != null && Files.isDirectory(p) && Files.isReadable(p);
}

try {
    checkpoints = storage.getAllCheckpoints(jobId);
} catch (CheckpointStorageException e) {
    log.error("Cannot list checkpoints for job {} (check perms/mount): {}", jobId, e.getMessage(), e);
    throw e; // recovery cannot proceed without checkpoint data
}

When it happens

Trigger: Calling getAllCheckpoints(jobId) when the job's checkpoint directory is unreadable (permission denied), the path is a file instead of a directory, an IO error occurs during directory traversal (e.g. disk error, symlink loop), or the underlying storage mount (NFS) becomes unavailable mid-listing.

Common situations: Running the Zeta engine as a different user than the one that wrote the checkpoints; checkpoint directory on a detached/unmounted NFS volume; directory permissions changed by an admin or cleanup job; storage parent path misconfigured so jobId resolves under a non-directory path.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• ${ExceptionUtils.getMessage(e)}
• No checkpoint found, job(${jobId}), pipeline(${pipelineId}),
• No checkpoint found for job ${jobId}
• No checkpoint found for job, job id is: ${jobId}
• Skipping checkpoint batch because none of its transactions r

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/d241097b154a1b90. Report an issue: GitHub.