apache/beam · error · IllegalArgumentException

Unsupported Beam field type %s for Snowflake column '%s'. Be

Error message

Unsupported Beam field type %s for Snowflake column '%s'. Beam DECIMAL does not include Snowflake precision and scale information.

What it means

Thrown by SnowflakeSchemaTransformUtils.toSnowflakeDataType when converting a Beam Schema field to a Snowflake column type and the field is a Beam DECIMAL. Snowflake's NUMBER type requires explicit precision and scale, but Beam's DECIMAL logical type in this path does not carry that metadata, so the connector refuses to guess and reports the limitation in the message details. This is a deliberate design boundary of the schema-mapping utility, not a data corruption issue.

Source

Thrown at sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/SnowflakeSchemaTransformUtils.java:316

      case FLOAT:
      case DOUBLE:
        return SnowflakeDouble.of();

      case STRING:
        return SnowflakeVarchar.of();

      case BOOLEAN:
        return SnowflakeBoolean.of();

      case BYTES:
        return SnowflakeBinary.of();

      case DATETIME:
        return SnowflakeTimestamp.of();

      case DECIMAL:
        throw unsupportedFieldType(
            field, "Beam DECIMAL does not include Snowflake precision and scale information.");

      case ARRAY:
      case ITERABLE:
      case MAP:
      case ROW:
      case LOGICAL_TYPE:
      default:
        throw unsupportedFieldType(field, null);
    }
  }

  public static IllegalArgumentException unsupportedFieldType(
      Schema.Field field, @Nullable String details) {
    String message =
        String.format(
            "Unsupported Beam field type %s for Snowflake column '%s'.",
            field.getType().getTypeName(), field.getName());

View on GitHub (pinned to 12126d8942)

Solutions

Change the Beam field type from DECIMAL to DOUBLE (FLOAT) in the schema before writing to Snowflake, accepting floating-point representation.
Re-declare the field as STRING or INT64 in the Beam schema and handle precision/scale yourself.
Cast the value upstream: in the source query cast DECIMAL columns to DOUBLE/VARCHAR so Beam infers a supported type.
If exact precision/scale is required, bypass toSnowflakeDataType and create/annotate the Snowflake table (NUMBER(p,s)) yourself, then write with a schema using supported types.

Example fix

// before
schema = Schema.of(Schema.Field.of("amount", Schema.FieldType.DECIMAL));

// after
schema = Schema.of(Schema.Field.of("amount", Schema.FieldType.DOUBLE)); // or cast in source: CAST(amount AS DOUBLE)

Defensive patterns

Strategy: validation

Validate before calling

// Check every field maps to a Snowflake type before attempting the write
static List<String> unmappableFields(Schema schema) {
  List<String> bad = new ArrayList<>();
  for (Schema.Field f : schema.getFields()) {
    switch (f.getType().getTypeName()) {
      case BYTE: case INT16: case INT32: case INT64: case FLOAT: case DOUBLE:
      case STRING: case BOOLEAN: case BYTES: case DATETIME:
        break;
      default: // includes DECIMAL, ARRAY, ITERABLE, MAP, ROW, LOGICAL_TYPE
        bad.add(f.getName() + ":" + f.getType().getTypeName());
    }
  }
  return bad;
}

Type guard

static boolean isSnowflakeMappable(Schema.Field field) {
  Schema.TypeName t = field.getType().getTypeName();
  return t == Schema.TypeName.BYTE || t == Schema.TypeName.INT16 || t == Schema.TypeName.INT32
      || t == Schema.TypeName.INT64 || t == Schema.TypeName.FLOAT || t == Schema.TypeName.DOUBLE
      || t == Schema.TypeName.STRING || t == Schema.TypeName.BOOLEAN
      || t == Schema.TypeName.BYTES || t == Schema.TypeName.DATETIME;
}

Try / catch

try {
  schema.getFields().forEach(SnowflakeSchemaTransformUtils::toSnowflakeDataType);
} catch (IllegalArgumentException e) {
  throw new IllegalStateException("Schema not writable to Snowflake, fix DECIMAL fields first: " + e.getMessage(), e);
}

Prevention

Avoid Schema.FieldType.DECIMAL in schemas destined for this connector; prefer DOUBLE or INT64.
Cast NUMERIC/DECIMAL source columns to DOUBLE/VARCHAR in the source query so Beam infers a mappable type.
Run the unmappableFields check on the schema during pipeline construction (fail fast at graph build time, not at runtime).
Pin schema definitions explicitly instead of relying on inference from Avro/JDBC sources that default to DECIMAL.

When it happens

Trigger: Calling toSnowflakeDataType (directly or via snowflakeType when generating a DDL statement for a staging table) with a Beam Schema.Field whose type is DECIMAL — typically produced when the inferred Beam schema of a PCollection contains NUMERIC/DECIMAL fields (e.g. from Avro, JDBC, or Calcite-inferred DECIMAL columns).

Common situations: Writing a Beam pipeline whose schema was inferred from sources that use DECIMAL/NUMERIC types (JDBC reads, Avro decimal logical types, BigQuery NUMERIC), then using SnowflakeIO schema transforms to write to Snowflake; Beam version upgrades that change type inference from DOUBLE to DECIMAL; hand-written schemas that use Schema.TypeName.DECIMAL assuming it maps to Snowflake NUMBER.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ce86062347043c94. Report an issue: GitHub.