apache/beam · error · RuntimeException

Type of field is unknown.

Error message

Type %s of field %s is unknown.

What it means

AwsTypes.fieldType maps AWS SDK marshalling types to Beam FieldTypes; when it encounters a type outside its mapping table it throws RuntimeException 'Type %s of field %s is unknown.' This means the AWS SDK field's marshalling type has no Beam schema representation in this version.

Solutions

1. Upgrade Apache Beam so AwsTypes knows the new marshalling type
2. Check the field type and exclude/flatten that field before schema conversion
3. Contribute a typeMapping entry via a custom schema provider subclass

Example fix

// before
Schema s = AwsSchemaUtils.schemaFor(UnsupportedAwsPojo.class);
// after
Schema s = Schema.builder()
    .addStringField("supportedField")
    // build only from fields with supported marshalling types
    .build();

for (SdkField<?> f : pojo.sdkFields()) { /* check f.marshallingType() is one of AwsTypes.typeMapping keys before schemaFor */ }

try { Schema s = AwsSchemaUtils.schemaFor(MyPojo.class); } catch (RuntimeException e) { /* fall back to manual field mapping */ }

When it happens

Trigger: Schema-for-ing an AWS SDK POJO containing a field whose MarshallingType has no mapping in AwsTypes.typeMapping (e.g. certain streaming/blob/document types); recursive fieldType/schemaFor traversal hitting an unmapped nested type.

Common situations: Newer AWS SDK versions introduce new marshalling types not yet mapped in the Beam version in use; using AwsSchemaProvider on POJOs with unusual field types.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• Unexpected marshalling type awsType
• Array schema is not properly formatted or unsupported
• Cannot select a subfield of a non-composite type.
• Could not infer beam type for thrift field
• Field type not matched.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/eeeaf79974a1b424. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/schemas/AwsTypes.java:92

          .build();

  private static FieldType fieldType(SdkField<?> field, Set<Class<?>> seen) {
    MarshallingType<?> type = field.marshallingType();
    if (type == LIST) {
      return FieldType.array(fieldType(elementField(field), seen));
    } else if (type == MAP) {
      return FieldType.map(FieldType.STRING, fieldType(valueField(field), seen));
    } else if (type == SDK_POJO) {
      SdkPojo builder = field.constructor().get();
      Class<?> clazz = targetClassOf(builder);
      checkState(!seen.contains(clazz), "Self-recursive types are not supported: %s", clazz);
      return FieldType.row(schemaFor(builder.sdkFields(), Sets.union(seen, singleton(clazz))));
    }
    FieldType fieldType = typeMapping.get(type);
    if (fieldType != null) {
      return fieldType;
    }
    throw new RuntimeException(
        String.format("Type %s of field %s is unknown.", type, normalizedNameOf(field)));
  }

  static FieldValueTypeInformation fieldValueTypeInformationFor(SdkField<?> sdkField) {
    TypeDescriptor<?> type = TypeDescriptor.of(sdkField.marshallingType().getTargetClass());
    return FieldValueTypeInformation.builder()
        .setName(normalizedNameOf(sdkField))
        .setType(type)
        .setRawType(sdkField.marshallingType().getClass())
        .setElementType(FieldValueTypeInformation.getIterableComponentType(type))
        .setMapKeyType(FieldValueTypeInformation.getMapKeyType(type))
        .setMapValueType(FieldValueTypeInformation.getMapValueType(type))
        .setOneOfTypes(Collections.emptyMap())
        .setNullable(true)
        .build();
  }

  private static Schema schemaFor(List<SdkField<?>> fields, Set<Class<?>> seen) {

View on GitHub (pinned to 12126d8942)