apache/iceberg · critical · IllegalStateException

Cannot process unknown snapshot operation: %s (snapshot id %

Error message

Cannot process unknown snapshot operation: %s (snapshot id %s)

What it means

BaseSparkMicroBatchPlanner.shouldProcess evaluates a snapshot's operation (append, replace, overwrite, delete...) against streaming options. The switch is exhaustive over known operations; if DataOperations returns a value the planner does not recognize, it throws an IllegalStateException with the operation name and snapshot id rather than silently skipping data.

Source

Thrown at spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/BaseSparkMicroBatchPlanner.java:76

        return true;
      case DataOperations.REPLACE:
        return false;
      case DataOperations.DELETE:
        Preconditions.checkState(
            readConf.streamingSkipDeleteSnapshots(),
            "Cannot process delete snapshot: %s, to ignore deletes, set %s=true",
            snapshot.snapshotId(),
            SparkReadOptions.STREAMING_SKIP_DELETE_SNAPSHOTS);
        return false;
      case DataOperations.OVERWRITE:
        Preconditions.checkState(
            readConf.streamingSkipOverwriteSnapshots(),
            "Cannot process overwrite snapshot: %s, to ignore overwrites, set %s=true",
            snapshot.snapshotId(),
            SparkReadOptions.STREAMING_SKIP_OVERWRITE_SNAPSHOTS);
        return false;
      default:
        throw new IllegalStateException(
            String.format(
                "Cannot process unknown snapshot operation: %s (snapshot id %s)",
                op.toLowerCase(Locale.ROOT), snapshot.snapshotId()));
    }
  }

  /**
   * Get the next snapshot skipping over rewrite and delete snapshots. Async must handle nulls.
   *
   * @param curSnapshot the current snapshot
   * @return the next valid snapshot (not a rewrite or delete snapshot), returns null if all
   *     remaining snapshots should be skipped.
   */
  protected Snapshot nextValidSnapshot(Snapshot curSnapshot) {
    Snapshot nextSnapshot;
    // if there were no valid snapshots, check for an initialOffset again
    if (curSnapshot == null) {
      StreamingOffset startingOffset =

View on GitHub (pinned to 86d9c8fc54)

Solutions

  1. Identify the writer that produced the snapshot (snapshot id is in the message) via metadata tables.
  2. Upgrade the Iceberg runtime on the reading Spark cluster to a version that understands the operation.
  3. As a workaround, skip past the offending snapshot by resetting the stream checkpoint to a later offset, accepting skipped data.
  4. If caused by custom tooling, fix it to emit a standard DataOperations value (e.g. 'append').
Defensive patterns

Strategy: validation

Validate before calling

// before starting the stream, inspect pending snapshot operations via metadata table
spark.read.format("iceberg").load(db + ".snapshots")
    .select("snapshot_id", "operation")
    .filter(!$"operation".isIn("append", "replace", "overwrite", "delete"))
    .show();

Try / catch

try {
    query.processAllAvailable();
} catch (IllegalStateException e) {
    if (e.getMessage().startsWith("Cannot process unknown snapshot operation")) {
        // upgrade reader Iceberg version or advance checkpoint past this snapshot
    }
}

Prevention

When it happens

Trigger: A streaming read encounters a snapshot whose DataOperations string is not one of append/replace/overwrite/delete — e.g. produced by a newer Iceberg writer, a custom engine, or future spec operation the deployed reader doesn't know — while nextValidSnapshot walks snapshots for the next micro-batch.

Common situations: Streaming from a table written by a much newer Iceberg version or a third-party writer emitting new operation types; custom catalog code setting a nonstandard operation string; reading a table after an engine upgrade where the reader version lags the writer.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/0ea93d197c997789. Report an issue: GitHub.