apache/iceberg · error · UncheckedIOException

Failed to write StreamingOffset to json

Error message

Failed to write StreamingOffset to json

What it means

StreamingOffset.json() serializes the offset (snapshotId, position, scanAllFiles) to a JSON string using a Jackson generator. Any IOException during generation is wrapped in UncheckedIOException with 'Failed to write StreamingOffset to json' and the cause preserved.

Source

Thrown at spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/StreamingOffset.java:99

    }

    return fromJsonNode(node);
  }

  @Override
  public String json() {
    StringWriter writer = new StringWriter();
    try (JsonGenerator generator = JsonUtil.factory().createGenerator(writer)) {
      generator.writeStartObject();
      generator.writeNumberField(VERSION, CURR_VERSION);
      generator.writeNumberField(SNAPSHOT_ID, snapshotId);
      generator.writeNumberField(POSITION, position);
      generator.writeBooleanField(SCAN_ALL_FILES, scanAllFiles);
      generator.writeEndObject();
      generator.flush();

    } catch (IOException e) {
      throw new UncheckedIOException("Failed to write StreamingOffset to json", e);
    }

    return writer.toString();
  }

  long snapshotId() {
    return snapshotId;
  }

  long position() {
    return position;
  }

  boolean shouldScanAllFiles() {
    return scanAllFiles;
  }

  @Override

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Check the wrapped cause; it is nearly always a classpath conflict — ensure a single consistent Jackson version.
2. Retry the query; if transient, the offset write will succeed on the next micro-batch.
3. If reproducible, report with the full stack trace; the serialization target is an in-memory StringWriter so repeated failure signals an environment problem.

try {
  String json = offset.json();
} catch (UncheckedIOException e) {
  logger.error("StreamingOffset serialization failed; check Jackson classpath", e);
}

When it happens

Trigger: An IOException occurs while writing JSON via the in-memory StringWriter-backed generator — extremely rare since the target is memory, but possible on internal generator failures.

Common situations: Rare JVM/Jackson internal failures during offset serialization when Spark persists offsets to the checkpoint directory; usually indicates a JVM resource issue or a Jackson version conflict on the classpath.

Understand the failure class

Background: "JSON serialization failed", "not JSON serializable", "Failed to serialize": why JSON marshaling errors happen and how to fix them — this error's family across 46 libraries.

Related errors

• Failed to parse StreamingOffset from JSON string ${json}
• Failed to read StreamingOffset from json
• Failed to write StreamingOffset to json
• Unsupported task type:
• Cannot write unknown type:

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/97e6b239b0698246. Report an issue: GitHub.