apache/seatunnel · error · CheckpointStorageException

Failed to serialize checkpoint data, state: ${state}

Error message

Failed to serialize checkpoint data, state: ${state}

What it means

HdfsStorage.storeCheckPoint serializes the PipelineState to bytes before writing to HDFS. If serialization throws an IOException, the method wraps it in a CheckpointStorageException whose message includes the failing state, pinpointing corrupt or unserializable checkpoint contents rather than a storage problem.

Source

Thrown at seatunnel-engine/seatunnel-engine-storage/checkpoint-storage-plugins/checkpoint-storage-hdfs/src/main/java/org/apache/seatunnel/engine/checkpoint/storage/hdfs/HdfsStorage.java:91

    }

    private Configuration getConfiguration(Map<String, String> config)
            throws CheckpointStorageException {
        String storageType =
                config.getOrDefault(STORAGE_TYPE_KEY, FileConfiguration.LOCAL.toString());
        config.remove(STORAGE_TYPE_KEY);
        AbstractConfiguration configuration =
                FileConfiguration.valueOf(storageType.toUpperCase()).getConfiguration();
        return configuration.buildConfiguration(config);
    }

    @Override
    public String storeCheckPoint(PipelineState state) throws CheckpointStorageException {
        byte[] datas;
        try {
            datas = serializeCheckPointData(state);
        } catch (IOException e) {
            throw new CheckpointStorageException(
                    String.format("Failed to serialize checkpoint data, state: %s", state), e);
        }
        Path filePath =
                new Path(
                        getStorageParentDirectory()
                                + state.getJobId()
                                + "/"
                                + getCheckPointName(state));

        Path tmpFilePath =
                new Path(
                        getStorageParentDirectory()
                                + state.getJobId()
                                + "/"
                                + getCheckPointName(state)
                                + STORAGE_TMP_SUFFIX);
        try (FSDataOutputStream out = fs.create(tmpFilePath, false)) {
            out.write(datas);

View on GitHub (pinned to cf67b549a7)

Solutions

1. Ensure all connector and engine jars are identical versions across the cluster
2. Check the nested IOException cause for ClassNotFoundException / NotSerializableException and add the missing jar or make the class serializable
3. Do not attempt to resume checkpoints written by an incompatible SeaTunnel version; use a fresh savepoint
4. Enable debug logging to inspect which field of PipelineState fails to serialize

try {
  storage.storeCheckPoint(state);
} catch (CheckpointStorageException e) {
  if (e.getMessage().startsWith("Failed to serialize checkpoint data")) {
    Throwable cause = e.getCause();
    if (cause instanceof IOException && cause.getCause() instanceof ClassNotFoundException) {
      log.error("Missing class in checkpoint state; align jar versions across cluster");
    }
  } else throw e;
}

When it happens

Trigger: storeCheckPoint(state) (also reached via modifyResumeTokenInCheckpoint) when PipelineState contains objects that fail Java serialization — e.g. states referencing classes not present on the classpath (ClassNotFoundException during serialization wrapped in IOException) or a corrupted in-memory state object.

Common situations: Version mismatch: checkpoints written/read across different SeaTunnel versions where a state class changed or was removed, connector plugin jars missing from the classpath, or mixing engine versions in a rolling upgrade.

Understand the failure class

Background: json.Marshal / "failed to marshal" errors in Go: why "unsupported type" happens and how to fix it — this error's family across 22 libraries.

Related errors

• Failed to get file system
• Failed to write checkpoint data, file: ${tmpFilePath}, state
• Failed to rename tmp file to final file
• No checkpoint found for job, job id is: ${jobId}
• Unsupported type in LocalTimeArrayType: ${eleSqlType}

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/8a9795165074a6ac. Report an issue: GitHub.