What causes "Failed to read StreamingOffset from json" in apache/iceberg?

Re-read from a fresh, valid InputStream containing JSON of form {"version":N,"snapshot_id":...,"position":...,"scan_all_files":...}

apache/iceberg · error · UncheckedIOException

Failed to read StreamingOffset from json

Error message

Failed to read StreamingOffset from json

What it means

StreamingOffset.fromJson(InputStream) throws UncheckedIOException with this message when reading the JSON node from the supplied InputStream fails with an IOException (stream read error, invalid encoding, etc.). The caller should close the stream after this method returns.

Source

Thrown at spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/StreamingOffset.java:80

    Preconditions.checkNotNull(json, "Cannot parse StreamingOffset JSON: null");

    try {
      JsonNode node = JsonUtil.mapper().readValue(json, JsonNode.class);
      return fromJsonNode(node);
    } catch (IOException e) {
      throw new UncheckedIOException(
          String.format("Failed to parse StreamingOffset from JSON string %s", json), e);
    }
  }

  static StreamingOffset fromJson(InputStream inputStream) {
    Preconditions.checkNotNull(inputStream, "Cannot parse StreamingOffset from inputStream: null");

    JsonNode node;
    try {
      node = JsonUtil.mapper().readValue(inputStream, JsonNode.class);
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to read StreamingOffset from json", e);
    }

    return fromJsonNode(node);
  }

  @Override
  public String json() {
    StringWriter writer = new StringWriter();
    try (JsonGenerator generator = JsonUtil.factory().createGenerator(writer)) {
      generator.writeStartObject();
      generator.writeNumberField(VERSION, CURR_VERSION);
      generator.writeNumberField(SNAPSHOT_ID, snapshotId);
      generator.writeNumberField(POSITION, position);
      generator.writeBooleanField(SCAN_ALL_FILES, scanAllFiles);
      generator.writeEndObject();
      generator.flush();

    } catch (IOException e) {

View on GitHub (pinned to 86d9c8fc54)

Solutions

Re-read from a fresh, valid InputStream containing JSON of form {"version":N,"snapshot_id":...,"position":...,"scan_all_files":...}
Repair or regenerate the checkpoint file, or restart the stream from a clean checkpoint
Check filesystem/network health for the underlying storage (HDFS/S3)

Example fix

null

Defensive patterns

Strategy: try-catch

Validate before calling

if (inputStream == null || inputStream.available() == 0) { throw new IllegalStateException("empty offset stream"); }

Try / catch

try {
  StreamingOffset offset = StreamingOffset.fromJson(stream);
} catch (UncheckedIOException e) {
  // fall back to reading offsets from the table / fresh checkpoint
}

Prevention

Always pass a fresh stream positioned at file start
Verify checkpoint files are fully written (no partial uploads) on the object store
Handle transient storage failures with retry before treating offsets as corrupt

When it happens

Trigger: Deserializing a StreamingOffset from an InputStream whose bytes are not valid JSON or that fails mid-read — e.g. a checkpoint file stream that is truncated or unreadable at recovery time.

Common situations: Corrupted checkpoint offset files; network/distributed filesystem transient failures while reading the offset stream; passing an already-consumed (empty) stream.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/9eda74a31cd7fcd4. Report an issue: GitHub.