{"record":{"id":"314756a076c86392","repo":"apache/iceberg","slug":"failed-writing-offset-to-s-314756","errorCode":null,"errorMessage":"Failed writing offset to: %s","messagePattern":"Failed writing offset to: (.+?)","errorType":"exception","errorClass":"UncheckedIOException","httpStatus":null,"severity":"critical","filePath":"spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/SparkMicroBatchStream.java","lineNumber":303,"sourceCode":"      }\n\n      table.refresh();\n      StreamingOffset offset = MicroBatchUtils.determineStartingOffset(table, fromTimestamp);\n\n      OutputFile outputFile = io.newOutputFile(initialOffsetLocation);\n      writeOffset(offset, outputFile);\n\n      return offset;\n    }\n\n    private void writeOffset(StreamingOffset offset, OutputFile file) {\n      try (OutputStream outputStream = file.create()) {\n        BufferedWriter writer =\n            new BufferedWriter(new OutputStreamWriter(outputStream, StandardCharsets.UTF_8));\n        writer.write(offset.json());\n        writer.flush();\n      } catch (IOException ioException) {\n        throw new UncheckedIOException(\n            String.format(\"Failed writing offset to: %s\", initialOffsetLocation), ioException);\n      }\n    }\n\n    private StreamingOffset readOffset(InputFile file) {\n      try (InputStream in = file.newStream()) {\n        return StreamingOffset.fromJson(in);\n      } catch (IOException ioException) {\n        throw new UncheckedIOException(\n            String.format(\"Failed reading offset from: %s\", initialOffsetLocation), ioException);\n      }\n    }\n  }\n}\n","sourceCodeStart":285,"sourceCodeEnd":318,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/SparkMicroBatchStream.java#L285-L318","documentation":"Wrapping error in SparkMicroBatchStream.writeOffset: persisting the initial StreamingOffset as JSON to the checkpoint location failed with an IOException, rethrown as UncheckedIOException naming the target location (%s). Common causes are an unwritable checkpoint path or storage outage.","triggerScenarios":"Failure creating or writing the offset file at initialOffsetLocation during Spark Structured Streaming micro-batch execution (storage down, permissions, missing directory).","commonSituations":"Checkpoint directory on S3/GCS/HDFS with permission issues; transient object-store outages; checkpoint path deleted or not writable.","solutions":["Check caused-by IOException for root cause","Verify checkpoint/offset location is writable and the parent directory exists","Fix storage credentials/permissions","Retry the streaming query after storage recovers"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// before starting the stream, ensure checkpoint parent dir is writable\nFiles/Path check or FS probe on checkpointLocation","typeGuard":null,"tryCatchPattern":"try { stream.writeOffset(offset); } catch (UncheckedIOException e) { e.getCause().printStackTrace(); /* alert and restart after storage recovery */ }","preventionTips":["Pre-create and verify checkpoint directory permissions","Use reliable storage with retry policies for checkpoints","Avoid sharing checkpoint locations across queries"],"tags":["spark","streaming","io-exception","checkpoint"],"backgroundTag":"file-write-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}