{"record":{"id":"2fff93cd0475dd65","repo":"apache/seatunnel","slug":"failed-to-read-checkpoint-data-file-name-is-fil","errorCode":null,"errorMessage":"Failed to read checkpoint data, file name is ${fileName},job id is ${jobId}","messagePattern":"Failed to read checkpoint data, file name is (.+?),job id is (.+?)","errorType":"exception","errorClass":"CheckpointStorageException","httpStatus":null,"severity":"error","filePath":"seatunnel-engine/seatunnel-engine-storage/checkpoint-storage-plugins/checkpoint-storage-hdfs/src/main/java/org/apache/seatunnel/engine/checkpoint/storage/hdfs/HdfsStorage.java","lineNumber":370,"sourceCode":"    }\n\n    /**\n     * Get checkpoint name\n     *\n     * @param fileName file name\n     * @return checkpoint data\n     */\n    private PipelineState readPipelineState(String fileName, String jobId)\n            throws CheckpointStorageException {\n        fileName =\n                getStorageParentDirectory() + jobId + DEFAULT_CHECKPOINT_FILE_PATH_SPLIT + fileName;\n        try (FSDataInputStream in = fs.open(new Path(fileName));\n                ByteArrayOutputStream stream = new ByteArrayOutputStream()) {\n            IOUtils.copyBytes(in, stream, 1024);\n            byte[] bytes = stream.toByteArray();\n            return deserializeCheckPointData(bytes);\n        } catch (IOException e) {\n            throw new CheckpointStorageException(\n                    String.format(\n                            \"Failed to read checkpoint data, file name is %s,job id is %s\",\n                            fileName, jobId),\n                    e);\n        }\n    }\n}\n","sourceCodeStart":352,"sourceCodeEnd":378,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-engine/seatunnel-engine-storage/checkpoint-storage-plugins/checkpoint-storage-hdfs/src/main/java/org/apache/seatunnel/engine/checkpoint/storage/hdfs/HdfsStorage.java#L352-L378","documentation":"readPipelineState opens a checkpoint file on HDFS via FSDataInputStream, copies bytes, and deserializes it; an IOException anywhere in that sequence is wrapped into CheckpointStorageException naming the file and job. Callers (getCheckpoint, getLatestCheckpoint, getAllCheckpoints, etc.) surface this when a checkpoint file exists but cannot be read or deserialized.","triggerScenarios":"Calling any checkpoint-read API when the target file was deleted mid-read, HDFS is unavailable, block read fails, or the byte content cannot be deserialized as checkpoint data (corrupt/truncated file).","commonSituations":"Corrupted checkpoint files after abrupt cluster shutdown; file deleted between listing and reading (race with cleanup); HDFS under-replicated/unavailable blocks; version mismatch where old serialized data cannot be deserialized by a newer build.","solutions":["Verify the file exists and is readable on HDFS and re-run the read; transient block/network failures often clear.","Check for corruption by inspecting the file (hdfs fsck) and re-run the job if the checkpoint is unrecoverable.","Ensure the seatunnel-engine version writing the checkpoint matches the one reading it (serialization compatibility).","Retry the read with backoff for transient IOExceptions from HDFS."],"exampleFix":"// before\nCheckpointData cp = storage.getCheckpoint(jobId, pipelineId, checkpointId);\n// after\nCheckpointData cp;\ntry {\n    cp = storage.getCheckpoint(jobId, pipelineId, checkpointId);\n} catch (CheckpointStorageException e) {\n    if (e.getMessage().startsWith(\"Failed to read checkpoint data\")) {\n        cp = storage.getLatestCheckpoint(jobId, pipelineId); // fall back to an intact checkpoint\n    } else throw e;\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    CheckpointData cp = storage.getCheckpoint(jobId, pipelineId, checkpointId);\n} catch (CheckpointStorageException e) {\n    if (e.getMessage().startsWith(\"Failed to read checkpoint data\")) {\n        // transient HDFS I/O or corrupt file: retry, then fall back to another checkpoint\n    } else { throw e; }\n}","preventionTips":["Avoid deleting checkpoint files while recovery may read them; coordinate cleanup schedules.","Monitor HDFS block health (hdfs fsck) to catch under-replicated/corrupt blocks early.","Keep writer and reader SeaTunnel versions consistent to avoid deserialization incompatibilities.","Gracefully shut down the cluster so checkpoint files are fully flushed."],"tags":["hdfs","io","deserialize","checkpoint"],"backgroundTag":"file-read-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}