apache/beam · error · java.lang.IllegalArgumentException

Snowflake row contains %d values, but the configured schema

Error message

Snowflake row contains %d values, but the configured schema contains %d fields.

What it means

toRow converts a raw Snowflake result row (String[] parts) into a Beam Row using the configured Schema. It first enforces arity: the number of values must equal the schema's field count. A mismatch means the query result shape diverges from the declared schema, so row construction is aborted with a format-level IllegalArgumentException.

Source

Thrown at sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/SnowflakeSchemaTransformUtils.java:175

  public static SnowflakeTableSchema toSnowflakeTableSchema(Schema schema) {
    SnowflakeColumn[] columns =
        schema.getFields().stream()
            .map(SnowflakeSchemaTransformUtils::toSnowflakeColumn)
            .toArray(SnowflakeColumn[]::new);

    return SnowflakeTableSchema.of(columns);
  }

  public static SnowflakeColumn toSnowflakeColumn(Schema.Field field) {
    SnowflakeDataType snowflakeType = toSnowflakeDataType(field);

    return SnowflakeColumn.of(field.getName(), snowflakeType, field.getType().getNullable());
  }

  public static Row toRow(String[] parts, Schema schema) {
    if (parts.length != schema.getFieldCount()) {
      throw new IllegalArgumentException(
          String.format(
              "Snowflake row contains %d values, but the configured schema contains %d fields.",
              parts.length, schema.getFieldCount()));
    }

    Row.Builder builder = Row.withSchema(schema);

    for (int i = 0; i < schema.getFieldCount(); i++) {
      Schema.Field field = schema.getField(i);
      builder.addValue(toBeamValue(parts[i], field));
    }

    return builder.build();
  }

  public static @Nullable Object toBeamValue(String value, Schema.Field field) {
    if (value == null || value.isEmpty()) {
      if (field.getType().getNullable()) {

View on GitHub (pinned to 12126d8942)

Solutions

Align the query's SELECT list with the configured schema (same columns, same order and count).
Regenerate/update the Schema (or TableSchema/Row coders) after any table change.
Validate schema against a sample row (parts.length vs schema.getFieldCount()) before running the pipeline.

Example fix

// before
Schema schema = Schema.of(Field.of("id", FieldType.STRING)); // row has 2 columns
Row row = SnowflakeSchemaTransformUtils.toRow(parts, schema);
// after
Schema schema = Schema.of(Field.of("id", FieldType.STRING), Field.of("name", FieldType.STRING));
Row row = SnowflakeSchemaTransformUtils.toRow(parts, schema);

Defensive patterns

Strategy: validation

Validate before calling

if (parts.length != schema.getFieldCount()) {
  throw new IllegalArgumentException("row arity mismatch: " + parts.length + " vs " + schema.getFieldCount());
}

Try / catch

try { Row r = SnowflakeSchemaTransformUtils.toRow(parts, schema); }
catch (IllegalArgumentException e) { deadLetter(rows, e); }

Prevention

Pin the query's SELECT column list to the schema; avoid SELECT *.
Re-derive the schema whenever the table/view changes.
Add a startup check comparing a sample row's length to schema.getFieldCount().

When it happens

Trigger: Running a query whose SELECT column count differs from the schema passed to the transform (added/dropped/renamed columns); a view or table altered after the schema was defined; streaming export producing extra trailing delimiter columns.

Common situations: Table schema changed upstream after pipeline config was frozen; user updated the query but not the withSchema()/row-descriptor config; wrong table selected for an existing schema.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/b0073a00884d6cf7. Report an issue: GitHub.