apache/iceberg · error · IllegalStateException

Unsupported streaming write mode: " + mode

Error message

Unsupported streaming write mode: " + mode

What it means

SparkWriteBuilder.toStreaming throws IllegalStateException when a streaming micro-batch asks Iceberg to write in a mode that is neither Append nor (filtered) Overwrite. Iceberg streaming sinks only support append and dynamic overwrite (alwaysTrue filter); any other BatchWrite mode is unsupported.

Source

Thrown at spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/SparkWriteBuilder.java:170

          return asDynamicOverwrite();
        } else if (mode instanceof CopyOnWriteOperation cow) {
          return asCopyOnWriteOperation(cow.scan(), cow.isolationLevel());
        } else {
          return asBatchAppend();
        }
      }

      @Override
      public StreamingWrite toStreaming() {
        if (mode instanceof OverwriteByFilter overwrite) {
          Preconditions.checkState(
              overwrite.expr() == Expressions.alwaysTrue(),
              "Unsupported streaming overwrite filter: " + overwrite.expr());
          return asStreamingOverwrite();
        } else if (mode == null || mode instanceof Append) {
          return asStreamingAppend();
        } else {
          throw new IllegalStateException("Unsupported streaming write mode: " + mode);
        }
      }
    };
  }

  private SparkWriteRequirements writeRequirements() {
    if (mode instanceof CopyOnWriteOperation cow) {
      return writeConf.copyOnWriteRequirements(cow.command());
    } else {
      return writeConf.writeRequirements();
    }
  }

  private void validateRowLineage() {
    Preconditions.checkArgument(
        writeIncludesRowLineage() || !writeNeedsRowLineage(),
        "Row lineage information is missing for write in mode: %s",
        mode);

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Use OutputMode.Append() for streaming writes to Iceberg
2. If you need overwrite semantics, use OutputMode.Update() with an alwaysTrue overwrite filter (dynamic overwrite)
3. Remove Complete() output mode — Iceberg streaming does not support it
4. Check foreachBatch code that may be calling the wrong writer variant

Example fix

// before
spark.writeStream().outputMode("complete").toTable("iceberg_table")
// after
spark.writeStream().outputMode("append").toTable("iceberg_table")

if (mode instanceof OutputMode.Complete()) {
  throw new IllegalArgumentException("Iceberg streaming sink supports only append or update-with-overwrite, not complete");
}

boolean isSupportedStreamingMode(OutputMode mode) {
  return mode == null || mode instanceof OutputMode.Append() ||
         (mode instanceof OutputMode.Update());
}

try { query = spark.writeStream().toTable("tbl").awaitTermination(); } catch (IllegalStateException e) { if (e.getMessage().contains("Unsupported streaming write mode")) { /* switch OutputMode to Append */ } else throw e; }

When it happens

Trigger: Spark structured streaming invokes SparkWriteBuilder's streaming writer with a mode object that is not Append, not Overwrite-with-alwaysTrue filter, and not null (e.g. Complete mode or a Delete mode passed via DataStreamWriter).

Common situations: Using .outputMode(OutputMode.Complete()) with foreachBatch/Iceberg sink; a custom streaming query that issues updates/deletes; upgrading Spark where a new write mode reaches the builder.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Deleted rows scan task is not supported yet
• seekToNewSource not supported
• Cannot add column %s since setting default values in Spark i
• Transform is not supported: ${transform}
• Cannot convert unknown expression: ${expr}

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/6d3132e75ddf213c. Report an issue: GitHub.