{"record":{"id":"cb5230db9f7c9e77","repo":"apache/beam","slug":"unsupported-beam-field-type-s-for-snowflake-column-s","errorCode":null,"errorMessage":"Unsupported Beam field type %s for Snowflake column '%s'.","messagePattern":"Unsupported Beam field type (.+?) for Snowflake column '(.+?)'\\.","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/SnowflakeSchemaTransformUtils.java","lineNumber":325,"sourceCode":"        return SnowflakeBoolean.of();\n\n      case BYTES:\n        return SnowflakeBinary.of();\n\n      case DATETIME:\n        return SnowflakeTimestamp.of();\n\n      case DECIMAL:\n        throw unsupportedFieldType(\n            field, \"Beam DECIMAL does not include Snowflake precision and scale information.\");\n\n      case ARRAY:\n      case ITERABLE:\n      case MAP:\n      case ROW:\n      case LOGICAL_TYPE:\n      default:\n        throw unsupportedFieldType(field, null);\n    }\n  }\n\n  public static IllegalArgumentException unsupportedFieldType(\n      Schema.Field field, @Nullable String details) {\n    String message =\n        String.format(\n            \"Unsupported Beam field type %s for Snowflake column '%s'.\",\n            field.getType().getTypeName(), field.getName());\n\n    if (details != null) {\n      message += \" \" + details;\n    }\n\n    return new IllegalArgumentException(message);\n  }\n}\n","sourceCodeStart":307,"sourceCodeEnd":343,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/SnowflakeSchemaTransformUtils.java#L307-L343","documentation":"Thrown by SnowflakeSchemaTransformUtils.toSnowflakeDataType when the Beam Schema field's type has no Snowflake mapping in this connector: ARRAY, ITERABLE, MAP, ROW, LOGICAL_TYPE, or any unrecognized default. The connector only maps scalar types (numbers, DOUBLE, VARCHAR, BOOLEAN, BINARY, TIMESTAMP) to Snowflake column types, so composite/nested fields are rejected with this message naming the Beam type and column.","triggerScenarios":"Calling toSnowflakeDataType (directly or via snowflakeType during DDL generation) with a Schema.Field whose type is ARRAY, ITERABLE, MAP, ROW, LOGICAL_TYPE, or an unknown TypeName — e.g. a Beam schema containing repeated fields, nested struct fields, maps, or custom logical types.","commonSituations":"PCollection schemas inferred from nested data formats (JSON, Avro nested records, Protobuf) that produce ROW/ARRAY/MAP fields; custom logical types registered by other IOs; pipelines written after a source schema change adds a nested column; users assuming Snowflake VARIANT/ARRAY support exists in this connector path.","solutions":["Flatten nested Beam ROW fields into separate scalar top-level fields in the schema before writing.","Serialize ARRAY/MAP/ROW fields to JSON strings (STRING type) and store them in a Snowflake VARCHAR or VARIANT column manually.","Drop unsupported nested fields from the schema with Select/apply pipeline transforms before the Snowflake write.","If Snowflake semi-structured storage is required, pre-materialize the flattened schema as the staging table yourself and map only supported scalar types through this connector."],"exampleFix":"// before\nSchema schema = Schema.of(\n    Schema.Field.of(\"id\", Schema.FieldType.INT64),\n    Schema.Field.of(\"tags\", Schema.FieldType.iterable(Schema.FieldType.STRING))); // throws\n\n// after\nSchema schema = Schema.of(\n    Schema.Field.of(\"id\", Schema.FieldType.INT64),\n    Schema.Field.of(\"tags_json\", Schema.FieldType.STRING)); // serialize list to JSON string first","handlingStrategy":"validation","validationCode":"// Ensure the schema contains only scalar types before the Snowflake write\nstatic void requireScalarSchema(Schema schema) {\n  for (Schema.Field f : schema.getFields()) {\n    Schema.TypeName t = f.getType().getTypeName();\n    if (t == Schema.TypeName.ARRAY || t == Schema.TypeName.ITERABLE\n        || t == Schema.TypeName.MAP || t == Schema.TypeName.ROW\n        || t == Schema.TypeName.LOGICAL_TYPE) {\n      throw new IllegalArgumentException(\"Field '\" + f.getName() + \"' uses unsupported type \" + t);\n    }\n  }\n}","typeGuard":"static boolean isScalarBeamType(Schema.FieldType type) {\n  switch (type.getTypeName()) {\n    case BYTE: case INT16: case INT32: case INT64: case FLOAT: case DOUBLE:\n    case STRING: case BOOLEAN: case BYTES: case DATETIME:\n      return true;\n    default:\n      return false;\n  }\n}","tryCatchPattern":"try {\n  schema.getFields().forEach(SnowflakeSchemaTransformUtils::toSnowflakeDataType);\n} catch (IllegalArgumentException e) {\n  LOG.error(\"Flatten/serialize nested fields before Snowflake write: {}\", e.getMessage());\n  throw e;\n}","preventionTips":["Flatten nested ROW structures with select()/flatten into scalar top-level fields before the Snowflake transform.","Serialize collections, maps, and nested records to JSON STRING columns.","Check the inferred schema of the PCollection (log schema.getFields()) during development before shipping the pipeline.","Remember this connector path maps only scalars — Snowflake VARIANT/ARRAY types are not produced by it, so don't assume nested support."],"tags":["snowflake","apache-beam","schema-mapping","unsupported-type"],"backgroundTag":"unsupported-dtype","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}