apache/beam · error · RuntimeException

Unexpected field type

Error message

Unexpected field type

What it means

AddFields.fillNewFields rebuilds each Row value while inserting the new field. Its switch over the field type has a default branch that throws 'Unexpected field type' because only ARRAY, ITERABLE, MAP, and ROW values can contain nested fields; any other type reaching this recursion is a bug or invalid configuration.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/transforms/AddFields.java:417

        case MAP:
          if (original == null) {
            return Collections.emptyMap();
          }
          Map<Object, Object> originalMap = (Map<Object, Object>) original;
          Map<Object, Object> filledMap = Maps.newHashMapWithExpectedSize(originalMap.size());
          Schema.FieldType mapValueType = fieldType.getMapValueType();
          AddFieldsInformation mapValueAddFieldInformation =
              addFieldsInformation.toBuilder().setOutputFieldType(mapValueType).build();
          for (Map.Entry<Object, Object> entry : originalMap.entrySet()) {
            filledMap.put(
                entry.getKey(),
                fillNewFields(entry.getValue(), mapValueType, mapValueAddFieldInformation));
          }
          return filledMap;

        default:
          throw new RuntimeException("Unexpected field type");
      }
    }

    @Override
    public PCollection<Row> expand(PCollection<T> input) {
      final AddFieldsInformation addFieldsInformation =
          getAddFieldsInformation(input.getSchema(), newFields);
      Schema outputSchema = checkNotNull(addFieldsInformation.getOutputFieldType().getRowSchema());

      return input
          .apply(
              ParDo.of(
                  new DoFn<T, Row>() {
                    @ProcessElement
                    public void processElement(@Element Row row, OutputReceiver<Row> o) {
                      o.output(fillNewFields(row, addFieldsInformation));
                    }
                  }))

View on GitHub (pinned to 12126d8942)

Solutions

1. Check that the parent of the field path is a composite type (ROW, MAP, ARRAY) in the input schema.
2. Ensure the FieldType passed to AddFields matches the schema; re-derive the schema after upstream transforms.
3. If the target is a scalar field, add the field at that level instead of descending.
4. If it appears despite a correct path, file/inspect against Beam version — this is an internal invariant default branch and may indicate a Beam bug.

Example fix

// before
rows.apply(AddFields.<Row>field("scalarField.inner", FieldType.STRING));
// after
rows.apply(AddFields.<Row>field("rowField.inner", FieldType.STRING));

Schema.Field parent = input.getSchema().getField("rowField");
if (parent.getType().getTypeName() != TypeName.ROW) {
  throw new IllegalStateException("fillNewFields expects ROW/MAP/ARRAY parent");
}

static boolean supportsNestedAdd(Schema.Field f) {
  switch (f.getType().getTypeName()) {
    case ROW: case MAP: case ARRAY: case ITERABLE: return true;
    default: return false;
  }
}

try { out = pc.apply(addFields); } catch (RuntimeException e) { if (e.getMessage().equals("Unexpected field type")) { /* inspect schema/path mismatch */ } else throw e; }

When it happens

Trigger: The AddFieldsInformation's resolved target type is not one of ARRAY/ITERABLE/MAP/ROW when fillNewFields is invoked (e.g. from newValue or recursively from fillNewFields), typically because the field path or declared new-field type mismatches the actual schema structure.

Common situations: Declaring AddFields.field("a.b", type) where 'a' resolves to a scalar in the resolved schema; internal inconsistency between getAddFieldsInformation's computed type and the actual row values, often after schema evolution or with custom SchemaCoder setups.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• We currently only support wildcards at terminal parts of sel
• Cannot select a subfield of a non-composite type.
• Timing number 0b" + timingNumber.toString(2) + " has more th
• No proto encoding for PaneInfoCoder, always part of Windowed
• Runner does not support draining.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/379e739af7548927. Report an issue: GitHub.