apache/seatunnel · warning

Some sink tables were skipped in Spark starter.

Error message

Some sink tables were skipped in Spark starter.

What it means

SinkExecuteProcessor in the Spark starter tracks tables that could not have a sink created (skippedTables) when building a multi-table sink pipeline. When only some tables were skipped, the job proceeds but this warning is logged listing each failed table, phase, plugin name, and the underlying error. It tells the developer that the submitted job will not write to every configured sink table.

Source

Thrown at seatunnel-core/seatunnel-spark-starter/seatunnel-spark-starter-common/src/main/java/org/apache/seatunnel/core/starter/spark/execution/SinkExecuteProcessor.java:209

            createdAnySink = true;
            String applicationId =
                    sparkRuntimeEnvironment.getStreamingContext().sparkContext().applicationId();
            CatalogTable[] catalogTables =
                    datasetTableInfo.getCatalogTables().toArray(new CatalogTable[0]);
            SparkSinkInjector.inject(
                            dataset.write(), sink, catalogTables, applicationId, parallelism)
                    .option("checkpointLocation", "/tmp")
                    .mode(SaveMode.Append)
                    .save();
        }
        if (!createdAnySink && !skippedTables.isEmpty()) {
            throw new TaskExecuteException(
                    MultiTableFailureHelper.formatFailedTableSummary(
                            "All candidate sink tables were skipped in Spark starter.",
                            skippedTables));
        }
        if (createdAnySink && !skippedTables.isEmpty()) {
            log.warn(
                    MultiTableFailureHelper.formatFailedTableSummary(
                            "Some sink tables were skipped in Spark starter.", skippedTables));
        }
        // the sink is the last stream
        return null;
    }

    public void handleSaveMode(SeaTunnelSink sink) {
        if (sink instanceof SupportSaveMode) {
            Optional<SaveModeHandler> saveModeHandler =
                    ((SupportSaveMode) sink).getSaveModeHandler();
            if (saveModeHandler.isPresent()) {
                try (SaveModeHandler handler = saveModeHandler.get()) {
                    handler.open();
                    new SaveModeExecuteWrapper(handler).execute();
                } catch (Exception e) {
                    throw new SeaTunnelRuntimeException(HANDLE_SAVE_MODE_FAILED, e);
                }

View on GitHub (pinned to cf67b549a7)

Solutions

1. Read the warn log's per-table summary to identify which table path/plugin/phase failed and fix that table's config.
2. Verify each sink table path exists and the sink plugin supports it in the Spark execution engine.
3. Re-run a batch with only the failing table to isolate the root error.
4. If skipping is unacceptable, check the multi-table skip/fail configuration option so failures abort the job instead of skipping.

Example fix

// before: one table path wrong among several
sink {
  Jdbc {
    source_table_name = "t1"
    database = "prod"
  }
}
// after: correct the path/plugin for the skipped table
sink {
  Jdbc {
    source_table_name = "t1"
    database = "prod"
    table = "orders"   // ensure target table exists and plugin supports it
  }
}

// Before submitting, validate each sink table path and plugin
sinkConfigs.forEach(cfg -> {
    String table = cfg.getString("source_table_name");
    if (!upstreamTables.contains(table)) {
        throw new IllegalArgumentException("Sink references unknown table: " + table);
    }
});

When it happens

Trigger: Executing a Spark job with multiple sink tables where creating/initializing the sink for one or more tables throws (bad table path, unsupported plugin per table, schema mismatch) while at least one sink was created successfully so the job continues.

Common situations: Multi-table configs where one target table was dropped or renamed, a sink plugin misconfigured for one database, or a schema drift making one target incompatible while other targets remain valid.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• All candidate sink tables were skipped in Spark starter.
• All candidate sink tables were skipped in Spark starter.
• Some sink tables were skipped in Flink starter.
• Some sink tables were skipped in Spark starter.
• Skip failed sink table in Spark starter: {}

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/889dd60a53f5899a. Report an issue: GitHub.