{"record":{"id":"0ea93d197c997789","repo":"apache/iceberg","slug":"cannot-process-unknown-snapshot-operation-s-sna-0ea93d","errorCode":null,"errorMessage":"Cannot process unknown snapshot operation: %s (snapshot id %s)","messagePattern":"Cannot process unknown snapshot operation: (.+?) \\(snapshot id (.+?)\\)","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"critical","filePath":"spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/BaseSparkMicroBatchPlanner.java","lineNumber":76,"sourceCode":"        return true;\n      case DataOperations.REPLACE:\n        return false;\n      case DataOperations.DELETE:\n        Preconditions.checkState(\n            readConf.streamingSkipDeleteSnapshots(),\n            \"Cannot process delete snapshot: %s, to ignore deletes, set %s=true\",\n            snapshot.snapshotId(),\n            SparkReadOptions.STREAMING_SKIP_DELETE_SNAPSHOTS);\n        return false;\n      case DataOperations.OVERWRITE:\n        Preconditions.checkState(\n            readConf.streamingSkipOverwriteSnapshots(),\n            \"Cannot process overwrite snapshot: %s, to ignore overwrites, set %s=true\",\n            snapshot.snapshotId(),\n            SparkReadOptions.STREAMING_SKIP_OVERWRITE_SNAPSHOTS);\n        return false;\n      default:\n        throw new IllegalStateException(\n            String.format(\n                \"Cannot process unknown snapshot operation: %s (snapshot id %s)\",\n                op.toLowerCase(Locale.ROOT), snapshot.snapshotId()));\n    }\n  }\n\n  /**\n   * Get the next snapshot skipping over rewrite and delete snapshots. Async must handle nulls.\n   *\n   * @param curSnapshot the current snapshot\n   * @return the next valid snapshot (not a rewrite or delete snapshot), returns null if all\n   *     remaining snapshots should be skipped.\n   */\n  protected Snapshot nextValidSnapshot(Snapshot curSnapshot) {\n    Snapshot nextSnapshot;\n    // if there were no valid snapshots, check for an initialOffset again\n    if (curSnapshot == null) {\n      StreamingOffset startingOffset =","sourceCodeStart":58,"sourceCodeEnd":94,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/BaseSparkMicroBatchPlanner.java#L58-L94","documentation":"BaseSparkMicroBatchPlanner.shouldProcess evaluates a snapshot's operation (append, replace, overwrite, delete...) against streaming options. The switch is exhaustive over known operations; if DataOperations returns a value the planner does not recognize, it throws an IllegalStateException with the operation name and snapshot id rather than silently skipping data.","triggerScenarios":"A streaming read encounters a snapshot whose DataOperations string is not one of append/replace/overwrite/delete — e.g. produced by a newer Iceberg writer, a custom engine, or future spec operation the deployed reader doesn't know — while nextValidSnapshot walks snapshots for the next micro-batch.","commonSituations":"Streaming from a table written by a much newer Iceberg version or a third-party writer emitting new operation types; custom catalog code setting a nonstandard operation string; reading a table after an engine upgrade where the reader version lags the writer.","solutions":["Identify the writer that produced the snapshot (snapshot id is in the message) via metadata tables.","Upgrade the Iceberg runtime on the reading Spark cluster to a version that understands the operation.","As a workaround, skip past the offending snapshot by resetting the stream checkpoint to a later offset, accepting skipped data.","If caused by custom tooling, fix it to emit a standard DataOperations value (e.g. 'append')."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// before starting the stream, inspect pending snapshot operations via metadata table\nspark.read.format(\"iceberg\").load(db + \".snapshots\")\n    .select(\"snapshot_id\", \"operation\")\n    .filter(!$\"operation\".isIn(\"append\", \"replace\", \"overwrite\", \"delete\"))\n    .show();","typeGuard":null,"tryCatchPattern":"try {\n    query.processAllAvailable();\n} catch (IllegalStateException e) {\n    if (e.getMessage().startsWith(\"Cannot process unknown snapshot operation\")) {\n        // upgrade reader Iceberg version or advance checkpoint past this snapshot\n    }\n}","preventionTips":["Keep writer and reader Iceberg versions compatible; upgrade readers before writers","Avoid custom engines/code writing nonstandard DataOperations values","Inspect the snapshots metadata table when pointing streams at externally-written tables"],"tags":["spark","streaming","snapshot","forward-compatibility"],"backgroundTag":"unsupported-enum-value","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}