{"record":{"id":"6d3132e75ddf213c","repo":"apache/iceberg","slug":"unsupported-streaming-write-mode-mode","errorCode":null,"errorMessage":"Unsupported streaming write mode: \" + mode","messagePattern":"Unsupported streaming write mode: \" \\+ mode","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/SparkWriteBuilder.java","lineNumber":170,"sourceCode":"          return asDynamicOverwrite();\n        } else if (mode instanceof CopyOnWriteOperation cow) {\n          return asCopyOnWriteOperation(cow.scan(), cow.isolationLevel());\n        } else {\n          return asBatchAppend();\n        }\n      }\n\n      @Override\n      public StreamingWrite toStreaming() {\n        if (mode instanceof OverwriteByFilter overwrite) {\n          Preconditions.checkState(\n              overwrite.expr() == Expressions.alwaysTrue(),\n              \"Unsupported streaming overwrite filter: \" + overwrite.expr());\n          return asStreamingOverwrite();\n        } else if (mode == null || mode instanceof Append) {\n          return asStreamingAppend();\n        } else {\n          throw new IllegalStateException(\"Unsupported streaming write mode: \" + mode);\n        }\n      }\n    };\n  }\n\n  private SparkWriteRequirements writeRequirements() {\n    if (mode instanceof CopyOnWriteOperation cow) {\n      return writeConf.copyOnWriteRequirements(cow.command());\n    } else {\n      return writeConf.writeRequirements();\n    }\n  }\n\n  private void validateRowLineage() {\n    Preconditions.checkArgument(\n        writeIncludesRowLineage() || !writeNeedsRowLineage(),\n        \"Row lineage information is missing for write in mode: %s\",\n        mode);","sourceCodeStart":152,"sourceCodeEnd":188,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/SparkWriteBuilder.java#L152-L188","documentation":"SparkWriteBuilder.toStreaming throws IllegalStateException when a streaming micro-batch asks Iceberg to write in a mode that is neither Append nor (filtered) Overwrite. Iceberg streaming sinks only support append and dynamic overwrite (alwaysTrue filter); any other BatchWrite mode is unsupported.","triggerScenarios":"Spark structured streaming invokes SparkWriteBuilder's streaming writer with a mode object that is not Append, not Overwrite-with-alwaysTrue filter, and not null (e.g. Complete mode or a Delete mode passed via DataStreamWriter).","commonSituations":"Using .outputMode(OutputMode.Complete()) with foreachBatch/Iceberg sink; a custom streaming query that issues updates/deletes; upgrading Spark where a new write mode reaches the builder.","solutions":["Use OutputMode.Append() for streaming writes to Iceberg","If you need overwrite semantics, use OutputMode.Update() with an alwaysTrue overwrite filter (dynamic overwrite)","Remove Complete() output mode — Iceberg streaming does not support it","Check foreachBatch code that may be calling the wrong writer variant"],"exampleFix":"// before\nspark.writeStream().outputMode(\"complete\").toTable(\"iceberg_table\")\n// after\nspark.writeStream().outputMode(\"append\").toTable(\"iceberg_table\")","handlingStrategy":"validation","validationCode":"if (mode instanceof OutputMode.Complete()) {\n  throw new IllegalArgumentException(\"Iceberg streaming sink supports only append or update-with-overwrite, not complete\");\n}","typeGuard":"boolean isSupportedStreamingMode(OutputMode mode) {\n  return mode == null || mode instanceof OutputMode.Append() ||\n         (mode instanceof OutputMode.Update());\n}","tryCatchPattern":"try { query = spark.writeStream().toTable(\"tbl\").awaitTermination(); } catch (IllegalStateException e) { if (e.getMessage().contains(\"Unsupported streaming write mode\")) { /* switch OutputMode to Append */ } else throw e; }","preventionTips":["Use OutputMode.Append for streaming writes to Iceberg","Avoid OutputMode.Complete with Iceberg sinks","For overwrite semantics in streaming, use forEachBatch with dynamic-overwrite (alwaysTrue filter)"],"tags":["spark","streaming","write-mode","unsupported"],"backgroundTag":"unsupported-operation","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}