apache/iceberg · critical · UncheckedIOException

Failed writing offset to: %s

Error message

Failed writing offset to: %s

What it means

Wrapping error in SparkMicroBatchStream.writeOffset: persisting the initial StreamingOffset as JSON to the checkpoint location failed with an IOException, rethrown as UncheckedIOException naming the target location (%s). Common causes are an unwritable checkpoint path or storage outage.

Source

Thrown at spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/SparkMicroBatchStream.java:303

      }

      table.refresh();
      StreamingOffset offset = MicroBatchUtils.determineStartingOffset(table, fromTimestamp);

      OutputFile outputFile = io.newOutputFile(initialOffsetLocation);
      writeOffset(offset, outputFile);

      return offset;
    }

    private void writeOffset(StreamingOffset offset, OutputFile file) {
      try (OutputStream outputStream = file.create()) {
        BufferedWriter writer =
            new BufferedWriter(new OutputStreamWriter(outputStream, StandardCharsets.UTF_8));
        writer.write(offset.json());
        writer.flush();
      } catch (IOException ioException) {
        throw new UncheckedIOException(
            String.format("Failed writing offset to: %s", initialOffsetLocation), ioException);
      }
    }

    private StreamingOffset readOffset(InputFile file) {
      try (InputStream in = file.newStream()) {
        return StreamingOffset.fromJson(in);
      } catch (IOException ioException) {
        throw new UncheckedIOException(
            String.format("Failed reading offset from: %s", initialOffsetLocation), ioException);
      }
    }
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Check caused-by IOException for root cause
2. Verify checkpoint/offset location is writable and the parent directory exists
3. Fix storage credentials/permissions
4. Retry the streaming query after storage recovers

// before starting the stream, ensure checkpoint parent dir is writable
Files/Path check or FS probe on checkpointLocation

try { stream.writeOffset(offset); } catch (UncheckedIOException e) { e.getCause().printStackTrace(); /* alert and restart after storage recovery */ }

When it happens

Trigger: Failure creating or writing the offset file at initialOffsetLocation during Spark Structured Streaming micro-batch execution (storage down, permissions, missing directory).

Common situations: Checkpoint directory on S3/GCS/HDFS with permission issues; transient object-store outages; checkpoint path deleted or not writable.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Failed reading offset from: %s
• Cannot load current offset at snapshot %d, the snapshot was
• Failed to parse StreamingOffset from JSON string %s
• Failed to parse StreamingOffset from JSON string %s
• Failed writing offset to: %s

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/314756a076c86392. Report an issue: GitHub.