apache/iceberg · error · UncheckedIOException

Failed to parse StreamingOffset from JSON string %s

Error message

Failed to parse StreamingOffset from JSON string %s

What it means

StreamingOffset.fromJson parses a JSON string into a StreamingOffset (used as a Spark micro-batch offset). If the string is not valid JSON or does not match the expected structure, the IOException (or structure failure) is wrapped in UncheckedIOException with the offending string in the message.

Source

Thrown at spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/StreamingOffset.java:68

   * @param snapshotId The current processed snapshot id.
   * @param position The position of last scanned file in snapshot.
   * @param scanAllFiles whether to scan all files in a snapshot; for example, to read all data when
   *     starting a stream.
   */
  StreamingOffset(long snapshotId, long position, boolean scanAllFiles) {
    this.snapshotId = snapshotId;
    this.position = position;
    this.scanAllFiles = scanAllFiles;
  }

  static StreamingOffset fromJson(String json) {
    Preconditions.checkNotNull(json, "Cannot parse StreamingOffset JSON: null");

    try {
      JsonNode node = JsonUtil.mapper().readValue(json, JsonNode.class);
      return fromJsonNode(node);
    } catch (IOException e) {
      throw new UncheckedIOException(
          String.format("Failed to parse StreamingOffset from JSON string %s", json), e);
    }
  }

  static StreamingOffset fromJson(InputStream inputStream) {
    Preconditions.checkNotNull(inputStream, "Cannot parse StreamingOffset from inputStream: null");

    JsonNode node;
    try {
      node = JsonUtil.mapper().readValue(inputStream, JsonNode.class);
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to read StreamingOffset from json", e);
    }

    return fromJsonNode(node);
  }

  @Override

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Inspect the printed JSON string in the message and correct/repair the checkpoint offset file
2. Restore the streaming query from a fresh checkpoint if the offset is unrecoverable
3. Verify both producer and consumer use the same Iceberg version so the offset schema matches
4. When parsing programmatically, validate the JSON structure before calling fromJson

Example fix

// before
StreamingOffset offset = StreamingOffset.fromJson(corruptedOffsetJson);
// after
StreamingOffset offset = StreamingOffset.fromJson(
    JsonUtil.mapper().writeValueAsString(
        Map.of("version", 1, "snapshot_id", snapshotId, "position", 0, "scan_all_files", false)));

if (json == null || json.isBlank() || !json.trim().startsWith("{")) {
  throw new IllegalArgumentException("Not a valid StreamingOffset JSON: " + json);
}

try {
  StreamingOffset offset = StreamingOffset.fromJson(json);
} catch (UncheckedIOException e) {
  // fall back to START_OFFSET or fail the batch with a checkpoint-repair path
  StreamingOffset offset = StreamingOffset.START_OFFSET;
}

When it happens

Trigger: Passing a malformed, truncated, or non-Iceberg JSON string to StreamingOffset.fromJson, e.g. a corrupted checkpoint offset value from the Spark streaming source.

Common situations: Corrupted or manually edited Spark streaming checkpoint directories; restoring checkpoints from a different Iceberg version whose offset schema differs; passing user-supplied strings during offset deserialization in tests.

Understand the failure class

Background: JSON parse error: "Unexpected token" / "not valid JSON" / "failed to parse" — what JSON parsers are really complaining about — this error's family across 45 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Cannot load current offset at snapshot %d, the snapshot was
• Failed writing offset to: %s
• Failed reading offset from: %s
• Failed to parse StreamingOffset from JSON string %s
• Failed writing offset to: %s

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/eeca7171fac6df47. Report an issue: GitHub.