What causes "Expecting exactly one field, found " in apache/beam?

Map the input to a schema with exactly one BYTES field, serializing other fields yourself (e.g. protobuf or beam Row.toBytes()).

apache/beam · error · IllegalArgumentException

Expecting exactly one field, found

Error message

Expecting exactly one field, found

What it means

TFRecordWriteSchemaTransformProvider.expand() requires the incoming PCollection's schema to contain exactly one field, since each element's single field is written as one TFRecord. If the input schema has any other number of fields, it throws this IllegalArgumentException naming the actual count.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/io/TFRecordWriteSchemaTransformProvider.java:144

      }
      if (configuration.getNumShards() > 0) {
        writeTransform = writeTransform.withNumShards(configuration.getNumShards());
      } else {
        writeTransform = writeTransform.withoutSharding();
      }
      if (Boolean.TRUE.equals(configuration.getNoSpilling())) {
        writeTransform = writeTransform.withNoSpilling();
      }
      if (configuration.getMaxNumWritersPerBundle() != null) {
        writeTransform =
            writeTransform.withMaxNumWritersPerBundle(configuration.getMaxNumWritersPerBundle());
      }

      // Obtain input schema and verify only one field and its bytes
      Schema inputSchema = input.get(INPUT).getSchema();
      int numFields = inputSchema.getFields().size();
      if (numFields != 1) {
        throw new IllegalArgumentException("Expecting exactly one field, found " + numFields);
      } else if (!inputSchema.getField(0).getType().equals(Schema.FieldType.BYTES)) {
        throw new IllegalArgumentException(
            "The input schema must have exactly one field of type byte.");
      }

      final String schemaField;
      if (inputSchema.getField(0).getName() != null) {
        schemaField = inputSchema.getField(0).getName();
      } else {
        schemaField = "record";
      }

      PCollection<Row> inputRows = input.get(INPUT);

      // Convert Beam Rows to byte arrays
      SerializableFunction<Row, byte[]> rowToBytesFn = getRowToBytesFn(schemaField);

      Schema errorSchema = ErrorHandling.errorSchema(inputSchema);

View on GitHub (pinned to 12126d8942)

Solutions

Map the input to a schema with exactly one BYTES field, serializing other fields yourself (e.g. protobuf or beam Row.toBytes()).
Verify the upstream PCollection's schema with pc.getSchema().getFieldCount() == 1 before applying.
If writing structured data, choose a SchemaTransform that supports multi-field formats (Parquet/Avro) instead.

Example fix

// before
PCollection<Row> rows = ...; // schema: name(str), payload(bytes)
rows.apply(TFRecordWriteSchemaTransformProvider...);

// after
PCollection<byte[]> single = rows.apply(MapElements.into(TypeDescriptor.of(byte[].class)).via(r -> r.getBytes("payload")));
single.apply(...); // schema has one BYTES field

Defensive patterns

Strategy: type-guard

Validate before calling

Schema s = pc.getSchema();
if (s.getFieldCount() != 1) {
  throw new IllegalArgumentException("TFRecord write needs exactly 1 field, got " + s.getFieldCount());
}

Type guard

static boolean isSingleBytesField(PCollection<?> pc) {
  Schema s = pc.getSchema();
  return s.getFieldCount() == 1 && s.getField(0).getType().equals(Schema.FieldType.BYTES);
}

Try / catch

try { pc.apply(tfRecordWrite); } catch (IllegalArgumentException e) { if (e.getMessage().startsWith("Expecting exactly one field")) { log.error("Input schema has {} fields; serialize to a single BYTES field first", pc.getSchema().getFieldCount()); } throw e; }

Prevention

Map all inputs to a single byte[] field before TFRecord write.
Assert schema shape in pipeline unit tests using PAssert on a sample schema.

When it happens

Trigger: Applying the TFRecord write SchemaTransform to a PCollection whose Schema has 0, 2, or more fields — e.g. a row type from a previous transform that wasn't reduced to a single bytes column.

Common situations: Feeding a multi-column table/Row schema directly into TFRecord write; forgetting to serialize rows into a single byte[] field first; SQL pipelines selecting multiple columns.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9505559f5489f522. Report an issue: GitHub.