apache/beam · error · IllegalArgumentException

table is required for batch writes.

Error message

table is required for batch writes.

What it means

SnowflakeWriteSchemaTransformProvider.expand() classifies a bounded input as a batch write, which must target a Snowflake table. If the table field is null or empty for a batch write, it throws this IllegalArgumentException during pipeline construction.

Solutions

1. Set table in the configuration to the target table, e.g. table: MY_DB.MY_SCHEMA.MY_TABLE.
2. If the write is meant to be streaming, ensure the input PCollection is unbounded and snowPipe is set instead.
3. Check the YAML/config key spelling so the table value actually reaches the provider.

Example fix

// before
config: {serverName: ..., database: ...} // batch input, no table
// after
config: {serverName: ..., database: ..., table: "MY_DB.MY_SCHEMA.MY_TABLE"}

boolean batch = pcollection.isBounded() == PCollection.IsBounded.BOUNDED;
if (batch && (config.getTable() == null || config.getTable().isEmpty())) {
  throw new IllegalArgumentException("Batch input requires table to be set");
}

try {
  return applyTransform(input, config);
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("table is required")) { /* add table config */ }
  throw e;
}

When it happens

Trigger: Applying the Snowflake write SchemaTransform to a bounded PCollection with the table config field omitted or empty (e.g. only snowPipe/staging settings supplied).

Common situations: Reusing a streaming-write config (which uses snowPipe, not table) for a batch pipeline, or the table key is misspelled/missing in the transform's YAML config.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• Either table or query must be specified.
• Failed to setLoginTimeout
• flushRowLimit must be greater than 0.
• flushTimeLimitMillis must be greater than 0.
• name + " cannot be empty"

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/61ab027ca430eaac. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/SnowflakeWriteSchemaTransformProvider.java:145

        Long flushTimeLimitMillis = configuration.getFlushTimeLimitMillis();
        if (flushTimeLimitMillis != null) {
          write = write.withFlushTimeLimit(Duration.millis(flushTimeLimitMillis));
        }

        Integer shardsNumber = configuration.getShardsNumber();
        if (shardsNumber != null) {
          write = write.withShardsNumber(shardsNumber);
        }

        String debugMode = configuration.getDebugMode();
        if (debugMode != null) {
          write = write.withDebugMode(parseStreamingLogLevel(debugMode));
        }
      } else {
        String table = configuration.getTable();

        if (table == null || table.isEmpty()) {
          throw new IllegalArgumentException("table is required for batch writes.");
        }

        write = write.to(table);

        String createDispositionValue = configuration.getCreateDisposition();

        if (createDispositionValue != null) {
          CreateDisposition createDisposition = parseCreateDisposition(createDispositionValue);

          write = write.withCreateDisposition(createDisposition);

          if (createDisposition == CreateDisposition.CREATE_IF_NEEDED) {
            write = write.withTableSchema(toSnowflakeTableSchema(rows.getSchema()));
          }
        }

        String writeDispositionValue = configuration.getWriteDisposition();

View on GitHub (pinned to 12126d8942)