apache/beam · error · IllegalArgumentException

'%s' does not support nested fields: %s

Error message

'%s' does not support nested fields: %s

What it means

RowFilter.verifyNoNestedFields throws IllegalArgumentException when keep(), drop(), or only() is given field names containing dots (nested fields), because those operations only support top-level fields. The message names the operation and lists the offending nested field names. Nested field selection requires a different API; this guard prevents silently wrong filtering.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/util/RowFilter.java:242

    return transformedSchema != null ? transformedSchema : rowSchema;
  }

  private void checkUnconfigured() {
    Preconditions.checkState(
        transformedSchema == null,
        "Invalid filter configuration: Please set only one of 'keep', 'drop', or 'only'.");
  }

  /** Verifies that this selection contains no nested fields. */
  private void verifyNoNestedFields(List<String> fields, String operation) {
    List<String> nestedFields = new ArrayList<>();
    for (String field : fields) {
      if (field.contains(".")) {
        nestedFields.add(field);
      }
    }
    if (!nestedFields.isEmpty()) {
      throw new IllegalArgumentException(
          String.format("'%s' does not support nested fields: %s", operation, nestedFields));
    }
  }

  /**
   * Checks whether a {@link Schema} contains a list of field names. Nested fields can be expressed
   * with dot-notation. Throws a helpful error in the case where a field doesn't exist, or if a
   * nested field could not be reached.
   */
  @VisibleForTesting
  static void validateSchemaContainsFields(
      Schema schema, List<String> specifiedFields, String operation) {
    Set<String> notFound = new HashSet<>();
    Set<String> notRowField = new HashSet<>();

    for (String field : specifiedFields) {
      List<String> levels = Splitter.on(".").splitToList(field);

View on GitHub (pinned to 12126d8942)

Solutions

1. Use only top-level field names with keep/drop/only.
2. Use RowFilter.keep()/drop() after flattening the schema, or operate on nested fields via the field-tree API (validateSchemaContainsFields supports nested paths).
3. Pre-flatten or explode nested rows before filtering.

Example fix

// before
rowFilter.only("user.name", "id");
// after
rowFilter.only("user", "id"); // top-level fields only

List<String> nested = fields.stream().filter(f -> f.contains(".")).collect(Collectors.toList());
if (!nested.isEmpty()) {
  throw new IllegalArgumentException("keep/drop/only do not support nested fields: " + nested);
}
rowFilter.keep(fields.toArray(new String[0]));

try {
  PCollection<Row> out = rowFilter.only(fields);
} catch (IllegalArgumentException e) {
  throw new IllegalArgumentException("Use top-level field names with only(): " + e.getMessage(), e);
}

When it happens

Trigger: rowFilter.keep("foo.bar"), rowFilter.drop("a.b"), or rowFilter.only("x.y") with any dotted field name; verifyNoNestedFields is called by keep/drop/only.

Common situations: Users familiar with nested-field support in validateSchemaContainsFields/keep paths mistakenly pass dotted paths to drop() or only(); schema evolution introduces nested fields where flat ones were expected.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Cannot provide a coder for a Beam Row. Please provide a sche
• Failed to decode Schema due to an error decoding Field proto
• Encountered UNSPECIFIED AtomicType
• Cannot select a subfield of a non-composite type.
• Convert requires a schema on the input.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/e284f79f9a0bd81e. Report an issue: GitHub.