apache/beam · error · IllegalArgumentException

Unsupported Beam field type %s for Snowflake column '%s'.

Error message

Unsupported Beam field type %s for Snowflake column '%s'.

What it means

Thrown by SnowflakeSchemaTransformUtils.toSnowflakeDataType when the Beam Schema field's type has no Snowflake mapping in this connector: ARRAY, ITERABLE, MAP, ROW, LOGICAL_TYPE, or any unrecognized default. The connector only maps scalar types (numbers, DOUBLE, VARCHAR, BOOLEAN, BINARY, TIMESTAMP) to Snowflake column types, so composite/nested fields are rejected with this message naming the Beam type and column.

Source

Thrown at sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/SnowflakeSchemaTransformUtils.java:325

        return SnowflakeBoolean.of();

      case BYTES:
        return SnowflakeBinary.of();

      case DATETIME:
        return SnowflakeTimestamp.of();

      case DECIMAL:
        throw unsupportedFieldType(
            field, "Beam DECIMAL does not include Snowflake precision and scale information.");

      case ARRAY:
      case ITERABLE:
      case MAP:
      case ROW:
      case LOGICAL_TYPE:
      default:
        throw unsupportedFieldType(field, null);
    }
  }

  public static IllegalArgumentException unsupportedFieldType(
      Schema.Field field, @Nullable String details) {
    String message =
        String.format(
            "Unsupported Beam field type %s for Snowflake column '%s'.",
            field.getType().getTypeName(), field.getName());

    if (details != null) {
      message += " " + details;
    }

    return new IllegalArgumentException(message);
  }
}

View on GitHub (pinned to 12126d8942)

Solutions

Flatten nested Beam ROW fields into separate scalar top-level fields in the schema before writing.
Serialize ARRAY/MAP/ROW fields to JSON strings (STRING type) and store them in a Snowflake VARCHAR or VARIANT column manually.
Drop unsupported nested fields from the schema with Select/apply pipeline transforms before the Snowflake write.
If Snowflake semi-structured storage is required, pre-materialize the flattened schema as the staging table yourself and map only supported scalar types through this connector.

Example fix

// before
Schema schema = Schema.of(
    Schema.Field.of("id", Schema.FieldType.INT64),
    Schema.Field.of("tags", Schema.FieldType.iterable(Schema.FieldType.STRING))); // throws

// after
Schema schema = Schema.of(
    Schema.Field.of("id", Schema.FieldType.INT64),
    Schema.Field.of("tags_json", Schema.FieldType.STRING)); // serialize list to JSON string first

Defensive patterns

Strategy: validation

Validate before calling

// Ensure the schema contains only scalar types before the Snowflake write
static void requireScalarSchema(Schema schema) {
  for (Schema.Field f : schema.getFields()) {
    Schema.TypeName t = f.getType().getTypeName();
    if (t == Schema.TypeName.ARRAY || t == Schema.TypeName.ITERABLE
        || t == Schema.TypeName.MAP || t == Schema.TypeName.ROW
        || t == Schema.TypeName.LOGICAL_TYPE) {
      throw new IllegalArgumentException("Field '" + f.getName() + "' uses unsupported type " + t);
    }
  }
}

Type guard

static boolean isScalarBeamType(Schema.FieldType type) {
  switch (type.getTypeName()) {
    case BYTE: case INT16: case INT32: case INT64: case FLOAT: case DOUBLE:
    case STRING: case BOOLEAN: case BYTES: case DATETIME:
      return true;
    default:
      return false;
  }
}

Try / catch

try {
  schema.getFields().forEach(SnowflakeSchemaTransformUtils::toSnowflakeDataType);
} catch (IllegalArgumentException e) {
  LOG.error("Flatten/serialize nested fields before Snowflake write: {}", e.getMessage());
  throw e;
}

Prevention

Flatten nested ROW structures with select()/flatten into scalar top-level fields before the Snowflake transform.
Serialize collections, maps, and nested records to JSON STRING columns.
Check the inferred schema of the PCollection (log schema.getFields()) during development before shipping the pipeline.
Remember this connector path maps only scalars — Snowflake VARIANT/ARRAY types are not produced by it, so don't assume nested support.

When it happens

Trigger: Calling toSnowflakeDataType (directly or via snowflakeType during DDL generation) with a Schema.Field whose type is ARRAY, ITERABLE, MAP, ROW, LOGICAL_TYPE, or an unknown TypeName — e.g. a Beam schema containing repeated fields, nested struct fields, maps, or custom logical types.

Common situations: PCollection schemas inferred from nested data formats (JSON, Avro nested records, Protobuf) that produce ROW/ARRAY/MAP fields; custom logical types registered by other IOs; pipelines written after a source schema change adds a nested column; users assuming Snowflake VARIANT/ARRAY support exists in this connector path.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/cb5230db9f7c9e77. Report an issue: GitHub.