{"record":{"id":"87d25b0d82a384f5","repo":"apache/beam","slug":"offsetretainer-failed-to-read-offset-from-delete-the-file-to","errorCode":null,"errorMessage":"OffsetRetainer: failed to read offset from . Delete the file to restart from the beginning.","messagePattern":"OffsetRetainer: failed to read offset from \\. Delete the file to restart from the beginning\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"critical","filePath":"sdks/java/io/debezium/src/main/java/org/apache/beam/io/debezium/FileSystemOffsetRetainer.java","lineNumber":120,"sourceCode":"   * Reads the offset JSON file and returns its contents, or {@code null} if the file does not yet\n   * exist (first run). Throws {@link RuntimeException} if the file exists but cannot be read, to\n   * prevent silently reprocessing data from the beginning.\n   */\n  @Override\n  public @Nullable Map<String, Object> loadOffset() {\n    try {\n      ResourceId resourceId = FileSystems.matchNewResource(path, /* isDirectory= */ false);\n      try (ReadableByteChannel channel = FileSystems.open(resourceId);\n          InputStream stream = Channels.newInputStream(channel)) {\n        Map<String, Object> offset = mapper().readValue(stream, MAP_TYPE);\n        LOG.info(\"OffsetRetainer: loaded offset from {}: {}\", path, offset);\n        return offset;\n      }\n    } catch (FileNotFoundException e) {\n      LOG.info(\"OffsetRetainer: no offset file found at {}; starting from the beginning.\", path);\n      return null;\n    } catch (IOException e) {\n      throw new RuntimeException(\n          \"OffsetRetainer: failed to read offset from \"\n              + path\n              + \". \"\n              + \"Delete the file to restart from the beginning.\",\n          e);\n    }\n  }\n\n  /**\n   * Serialises {@code offset} to JSON and writes it atomically to the configured path.\n   *\n   * <p>If the offset is identical to the last successfully written one, the write is skipped to\n   * avoid unnecessary I/O on every checkpoint.\n   *\n   * <p>Otherwise the data is first written to a {@code .tmp} sibling file and then renamed to the\n   * final path, so a mid-write crash leaves the previous offset intact.\n   *\n   * <p>Errors are logged as warnings and swallowed so the pipeline continues.","sourceCodeStart":102,"sourceCodeEnd":138,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/debezium/src/main/java/org/apache/beam/io/debezium/FileSystemOffsetRetainer.java#L102-L138","documentation":"FileSystemOffsetRetainer.loadOffset reads the persisted offset file so a resumable Debezium pipeline can restart where it stopped. A FileNotFoundException is benign (start from beginning), but any other IOException while reading is wrapped in this RuntimeException because a corrupt/unreadable offset file must not be silently ignored.","triggerScenarios":"The offset file at `path` exists but cannot be read: permission denied, I/O error on the filesystem, or a truncated/corrupt file causing the deserializing read to fail with IOException.","commonSituations":"Shared/NFS mount flakiness, offset file left unreadable after a job ran as a different service account, disk-full or hardware failure during a previous saveOffset.","solutions":["Delete the offset file at the logged path to restart consumption from the beginning (as the message suggests).","Fix filesystem permissions or mount health so the runner's service account can read the file.","If the offset is valuable, inspect/repair the file rather than deleting, then re-run.","Make the retention directory reliable (local persistent disk or healthy GCS/HDFS mount)."],"exampleFix":"// shell\nrm /var/beam/debezium-offset/offset.json  # then re-launch the pipeline","handlingStrategy":"fallback","validationCode":"File offsetFile = new File(path);\nif (offsetFile.exists() && !offsetFile.canRead()) {\n  LOG.warn(\"Offset file unreadable; will restart from beginning\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  pipeline.run();\n} catch (RuntimeException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"OffsetRetainer: failed to read offset\")) {\n    new File(offsetPath).delete(); // restart from beginning\n    pipeline.run();\n  } else throw e;\n}","preventionTips":["Run all pipeline workers under the same service account that owns the offset file","Use a reliable, healthy storage backend for offset retention","Monitor disk health/mount state before resuming jobs","Back up the offset file before infrastructure maintenance"],"tags":["java","io","filesystem","state-restore","apache-beam"],"backgroundTag":"file-read-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}