apache/iceberg · error · UncheckedIOException

Failed to write StreamingOffset to json

Error message

Failed to write StreamingOffset to json

What it means

Wrapping error in StreamingOffset.json: serializing the offset's fields (version, snapshotId, position, scanAllFiles) to JSON via the Jackson generator threw an IOException, rethrown as UncheckedIOException. Because output is a StringWriter, an IO failure here is essentially a serialization bug, not a storage problem.

Solutions

1. Inspect the wrapped cause for the generator failure
2. Retry the serialization — transient failures typically resolve
3. Verify the offset's snapshotId/position values are sane (no corrupt longs) before serialization
4. Report to Iceberg if it recurs, since this path should not fail under normal conditions

try {
  String json = offset.json();
} catch (UncheckedIOException e) {
  throw new RuntimeException("Offset serialization failed; abort batch commit", e);
}

When it happens

Trigger: Calling StreamingOffset.json when the internal JsonFactory/generator fails (e.g. I/O error writing to the underlying writer, or invalid field values during generation).

Common situations: Rare JVM/Jackson-level failures during offset serialization when committing micro-batch progress; custom subclasses writing invalid fields.

Understand the failure class

Background: "JSON serialization failed", "not JSON serializable", "Failed to serialize": why JSON marshaling errors happen and how to fix them — this error's family across 46 libraries.

Related errors

• Cannot load current offset at snapshot
• Cannot load current offset at snapshot
• Cannot load current offset at snapshot
• Cannot process unknown snapshot operation
• Cannot process unknown snapshot operation

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/8ed0a9d12f92ef06. Report an issue: GitHub.

Appendix: source

Thrown at spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/StreamingOffset.java:99

    }

    return fromJsonNode(node);
  }

  @Override
  public String json() {
    StringWriter writer = new StringWriter();
    try (JsonGenerator generator = JsonUtil.factory().createGenerator(writer)) {
      generator.writeStartObject();
      generator.writeNumberField(VERSION, CURR_VERSION);
      generator.writeNumberField(SNAPSHOT_ID, snapshotId);
      generator.writeNumberField(POSITION, position);
      generator.writeBooleanField(SCAN_ALL_FILES, scanAllFiles);
      generator.writeEndObject();
      generator.flush();

    } catch (IOException e) {
      throw new UncheckedIOException("Failed to write StreamingOffset to json", e);
    }

    return writer.toString();
  }

  long snapshotId() {
    return snapshotId;
  }

  long position() {
    return position;
  }

  boolean shouldScanAllFiles() {
    return scanAllFiles;
  }

  @Override

View on GitHub (pinned to 86d9c8fc54)