apache/beam · error · RuntimeException

Unexpected field descriptor type.

Error message

Unexpected field descriptor type.

What it means

DropFields.complement walks nested field descriptors to compute the complement of the selected fields. While descending, the field type is expected to be a composite (ROW), ARRAY/LIST, or MAP; any other type name encountered while navigating a nested FieldAccessDescriptor is a bug in how the descriptor was constructed, so it throws RuntimeException.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/transforms/DropFields.java:117

            input.getNestedFieldsAccessed().keySet().stream()
                .collect(Collectors.toMap(k -> k.getFieldId(), k -> k));

        FieldAccessDescriptor.FieldDescriptor fieldDescriptor = nestedFields.get(i);
        if (fieldDescriptor != null) {
          // Some subfields are selected, so recursively calculate the complementary subfields to
          // select.
          FieldType fieldType = inputSchema.getField(i).getType();
          for (FieldAccessDescriptor.FieldDescriptor.Qualifier qualifier :
              fieldDescriptor.getQualifiers()) {
            switch (qualifier.getKind()) {
              case LIST:
                fieldType = fieldType.getCollectionElementType();
                break;
              case MAP:
                fieldType = fieldType.getMapValueType();
                break;
              default:
                throw new RuntimeException("Unexpected field descriptor type.");
            }
          }
          checkArgument(fieldType.getTypeName().isCompositeType());
          FieldAccessDescriptor nestedDescriptor =
              input.getNestedFieldsAccessed().get(fieldDescriptor);
          nestedFieldsToSelect.put(
              fieldDescriptor, complement(fieldType.getRowSchema(), nestedDescriptor));
        } else {
          // Neither the field nor the subfield is selected. This means we should select it.
          fieldNamesToSelect.add(field.getName());
        }
      }

      FieldAccessDescriptor fieldAccess = FieldAccessDescriptor.withFieldNames(fieldNamesToSelect);
      for (Map.Entry<FieldAccessDescriptor.FieldDescriptor, FieldAccessDescriptor> entry :
          nestedFieldsToSelect.entrySet()) {
        fieldAccess = fieldAccess.withNestedField(entry.getKey(), entry.getValue());
      }

View on GitHub (pinned to 12126d8942)

Solutions

1. Fix the field path so nested access only occurs on ROW/ARRAY/MAP typed fields.
2. Verify the input schema field types before building the FieldAccessDescriptor.
3. Check for schema drift: if the pipeline schema changed, update the DropFields paths.
4. If it appears with a valid descriptor, file a Beam bug — this indicates an internal inconsistency in descriptor resolution.

Example fix

// before
.withFields("scalarField.nested") // scalarField is INT64
// after
.withFields("rowField.nested")

Schema.FieldType t = schema.getField(path[0]).getType(); if (!(t.getTypeName().isCompositeType() || t.getTypeName().equals(TypeName.ARRAY) || t.getTypeName().equals(TypeName.MAP))) throw new IllegalArgumentException("Cannot descend into non-composite field");

try { result = input.apply(DropFields.withoutFields(desc)); } catch (RuntimeException e) { if (e.getMessage().equals("Unexpected field descriptor type.")) { /* fix descriptor paths */ } throw e; }

When it happens

Trigger: Passing a FieldAccessDescriptor with nested field accesses that descend through a field whose type is not ROW, ARRAY, or MAP at the level being navigated — e.g. referencing nested fields of a primitive-typed field.

Common situations: Typing a wrong field path in withFields/withoutFields (accessing .subfield on a non-composite column), or schema drift where a field changed from a row/array/map to a scalar between versions.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• ${resolved} references multiple fields.
• Timing number 0b" + timingNumber.toString(2) + " has more th
• No proto encoding for PaneInfoCoder, always part of Windowed
• Runner does not support draining.
• cannot encode a null Integer

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f24cd5ed97dc0ce9. Report an issue: GitHub.