apache/iceberg · error · UncheckedIOException

Failed to read StreamingOffset from json

Error message

Failed to read StreamingOffset from json

What it means

StreamingOffset.fromJson(InputStream) reads JSON from a stream and parses it into a StreamingOffset. Any IOException during reading the stream is wrapped in UncheckedIOException with this generic message and the original cause attached.

Source

Thrown at spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/StreamingOffset.java:80

    Preconditions.checkNotNull(json, "Cannot parse StreamingOffset JSON: null");

    try {
      JsonNode node = JsonUtil.mapper().readValue(json, JsonNode.class);
      return fromJsonNode(node);
    } catch (IOException e) {
      throw new UncheckedIOException(
          String.format("Failed to parse StreamingOffset from JSON string %s", json), e);
    }
  }

  static StreamingOffset fromJson(InputStream inputStream) {
    Preconditions.checkNotNull(inputStream, "Cannot parse StreamingOffset from inputStream: null");

    JsonNode node;
    try {
      node = JsonUtil.mapper().readValue(inputStream, JsonNode.class);
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to read StreamingOffset from json", e);
    }

    return fromJsonNode(node);
  }

  @Override
  public String json() {
    StringWriter writer = new StringWriter();
    try (JsonGenerator generator = JsonUtil.factory().createGenerator(writer)) {
      generator.writeStartObject();
      generator.writeNumberField(VERSION, CURR_VERSION);
      generator.writeNumberField(SNAPSHOT_ID, snapshotId);
      generator.writeNumberField(POSITION, position);
      generator.writeBooleanField(SCAN_ALL_FILES, scanAllFiles);
      generator.writeEndObject();
      generator.flush();

    } catch (IOException e) {

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Check the wrapped cause for the underlying I/O failure (permissions, network, truncation)
2. Repair or restore the checkpoint offset file from backup
3. Ensure the stream is fully readable and not closed before parsing
4. Restart the streaming query with a valid checkpoint if the offset cannot be recovered

Example fix

// before
try (InputStream in = new FileInputStream(maybeMissingOffsetFile)) {
  offset = StreamingOffset.fromJson(in);
}
// after
if (!offsetFile.exists()) {
  offset = StreamingOffset.START_OFFSET;
} else {
  try (InputStream in = new FileInputStream(offsetFile)) {
    offset = StreamingOffset.fromJson(in);
  }
}

if (inputStream == null || inputStream.available() <= 0) {
  return StreamingOffset.START_OFFSET;
}

try {
  offset = StreamingOffset.fromJson(in);
} catch (UncheckedIOException e) {
  LOG.warn("Unreadable offset stream, resetting", e);
  offset = StreamingOffset.START_OFFSET;
}

When it happens

Trigger: Supplying a closed, unreadable, or failing InputStream to StreamingOffset.fromJson(InputStream), e.g. a truncated checkpoint offset file.

Common situations: Reading offsets from corrupted checkpoint storage (HDFS/S3 failures); file truncated mid-write; permission errors opening the offset stream.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Failed to parse StreamingOffset from JSON string %s
• Failed to read StreamingOffset from json
• Failed to write StreamingOffset to json
• Interrupted while polling queue
• Table refresh failed

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/b65d8750719e29aa. Report an issue: GitHub.