apache/beam · error · RuntimeException

Cannot select a subfield of a non-composite type.

Error message

Cannot select a subfield of a non-composite type.

What it means

AddFields.getAddFieldsInformation recursively computes the output type when adding fields to a nested schema. When it reaches a type that has no composite structure (not ARRAY, ITERABLE, MAP, or ROW), there is no subfield to descend into, so it throws a RuntimeException from the switch's default branch.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/transforms/AddFields.java:340

          fieldType = Schema.FieldType.array(addFieldsInformation.getOutputFieldType());
          break;

        case ITERABLE:
          addFieldsInformation =
              getAddFieldsInformation(inputFieldType.getCollectionElementType(), nestedFields);
          fieldType = Schema.FieldType.iterable(addFieldsInformation.getOutputFieldType());
          break;

        case MAP:
          addFieldsInformation =
              getAddFieldsInformation(inputFieldType.getMapValueType(), nestedFields);
          fieldType =
              Schema.FieldType.map(
                  inputFieldType.getMapKeyType(), addFieldsInformation.getOutputFieldType());
          break;

        default:
          throw new RuntimeException("Cannot select a subfield of a non-composite type.");
      }
      fieldType = fieldType.withNullable(inputFieldType.getNullable());
      return addFieldsInformation.toBuilder().setOutputFieldType(fieldType).build();
    }

    private static Row fillNewFields(Row row, AddFieldsInformation addFieldsInformation) {
      Schema outputSchema = checkNotNull(addFieldsInformation.getOutputFieldType().getRowSchema());

      List<Object> newValues = Lists.newArrayListWithCapacity(outputSchema.getFieldCount());
      for (int i = 0; i < row.getFieldCount(); ++i) {
        AddFieldsInformation nested = addFieldsInformation.getNestedNewValues().get(i);
        if (nested != null) {
          // New fields were added to nested subfields of this value. Recursively fill them out
          // before adding to the new row.
          Object newValue = fillNewFields(row.getValue(i), nested.getOutputFieldType(), nested);
          newValues.add(newValue);
        } else {
          // Nothing changed. Just copy the old value into the new row.

View on GitHub (pinned to 12126d8942)

Solutions

  1. Correct the field path so it only descends through composite (ROW/MAP/ARRAY) types.
  2. Verify the input schema with pcollection.getSchema() and confirm the parent field is a ROW/MAP/ARRAY before adding a nested field.
  3. If you want to add a top-level field, use a non-nested name like 'a_b' instead of 'a.b'.
  4. Wrap the transform application in a try-catch for RuntimeException if schemas are dynamic, and fail gracefully.

Example fix

// before (a is a STRING field)
PCollection<Row> out = rows.apply(AddFields.<Row>field("a.b", FieldType.STRING));
// after
PCollection<Row> out = rows.apply(AddFields.<Row>field("a_row.b", FieldType.STRING));
Defensive patterns

Strategy: validation

Validate before calling

Schema schema = pc.getSchema();
Schema.Field f = schema.getField("a");
TypeName tn = f.getType().getTypeName();
if (!EnumSet.of(TypeName.ROW, TypeName.MAP, TypeName.ARRAY, TypeName.ITERABLE).contains(tn)) {
  throw new IllegalStateException("cannot descend into non-composite field 'a'");
}

Type guard

static boolean isComposite(Schema.Field f) {
  switch (f.getType().getTypeName()) {
    case ROW: case MAP: case ARRAY: case ITERABLE: return true;
    default: return false;
  }
}

Try / catch

try { out = pc.apply(AddFields.<Row>field(path, type)); } catch (RuntimeException e) { if (e.getMessage().contains("non-composite")) { /* fix path or skip */ } else throw e; }

Prevention

When it happens

Trigger: Calling AddFields.<FieldName> or AddFields.field with a field path that traverses INTO a scalar leaf type, e.g. adding 'a.b' where field 'a' is a STRING or INT64 rather than a ROW/MAP/ARRAY.

Common situations: Typos in nested field paths (intending 'a.b' on a row schema but 'a' is actually scalar), or schemas changed upstream so a formerly-ROW field became a primitive, making the path invalid.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/91587910ec9ed905. Report an issue: GitHub.