apache/iceberg · error · UncheckedIOException

Failed to parse StreamingOffset from JSON string ${json}

Error message

Failed to parse StreamingOffset from JSON string ${json}

What it means

StreamingOffset.fromJson(String) deserializes a Spark Structured Streaming offset from JSON. If Jackson cannot parse the string, the IOException is wrapped in UncheckedIOException with the offending JSON in the message, preserving the cause.

Source

Thrown at spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/StreamingOffset.java:68

   * @param snapshotId The current processed snapshot id.
   * @param position The position of last scanned file in snapshot.
   * @param scanAllFiles whether to scan all files in a snapshot; for example, to read all data when
   *     starting a stream.
   */
  StreamingOffset(long snapshotId, long position, boolean scanAllFiles) {
    this.snapshotId = snapshotId;
    this.position = position;
    this.scanAllFiles = scanAllFiles;
  }

  static StreamingOffset fromJson(String json) {
    Preconditions.checkNotNull(json, "Cannot parse StreamingOffset JSON: null");

    try {
      JsonNode node = JsonUtil.mapper().readValue(json, JsonNode.class);
      return fromJsonNode(node);
    } catch (IOException e) {
      throw new UncheckedIOException(
          String.format("Failed to parse StreamingOffset from JSON string %s", json), e);
    }
  }

  static StreamingOffset fromJson(InputStream inputStream) {
    Preconditions.checkNotNull(inputStream, "Cannot parse StreamingOffset from inputStream: null");

    JsonNode node;
    try {
      node = JsonUtil.mapper().readValue(inputStream, JsonNode.class);
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to read StreamingOffset from json", e);
    }

    return fromJsonNode(node);
  }

  @Override

View on GitHub (pinned to 86d9c8fc54)

Solutions

Inspect and repair the offending JSON in the checkpoint offsets file (message includes the raw string).
Reset/rebuild the streaming checkpoint if the offset is unrecoverable (restart from an earlier checkpoint or trigger once).
Verify the offset JSON matches the expected shape {version, snapshot-id, position, scan-all-files}.

Example fix

// before
StreamingOffset offset = StreamingOffset.fromJson("{snapshot-id: 1,"); // malformed

// after
String json = Files.readString(offsetPath);
Preconditions.checkArgument(json.trim().startsWith("{"), "Not JSON: %s", json);
StreamingOffset offset = StreamingOffset.fromJson(json);

Defensive patterns

Strategy: try-catch

Validate before calling

String json = readOffsetFile(path);
if (json == null || !json.trim().startsWith("{")) {
  throw new IllegalStateException("Corrupt StreamingOffset JSON at " + path);
}

Try / catch

try {
  StreamingOffset offset = StreamingOffset.fromJson(json);
} catch (UncheckedIOException e) {
  // recover from an earlier checkpoint offset or reset the checkpoint
}

Prevention

Never hand-edit checkpoint offset files.
Back up the streaming checkpoint directory.
Keep the Iceberg version stable across query restarts.

When it happens

Trigger: Passing a malformed or truncated JSON string to StreamingOffset.fromJson(String) — e.g. a corrupted offset in the streaming checkpoint's offsets/ directory, or hand-edited offset JSON.

Common situations: Corrupted or partially-written checkpoint offset files after a crash; manually editing offsets; reading offsets produced by a different Iceberg version with a changed schema.

Understand the failure class

Background: JSON parse error: "Unexpected token" / "not valid JSON" / "failed to parse" — what JSON parsers are really complaining about — this error's family across 45 libraries.

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/1809a3cc21601640. Report an issue: GitHub.