apache/beam · error · IllegalArgumentException

The input schema must have exactly one field of type byte.

Error message

The input schema must have exactly one field of type byte.

What it means

After confirming the input schema has exactly one field, TFRecordWriteSchemaTransformProvider.expand() checks that this field's type is BYTES, because TFRecord elements must be raw byte arrays. A single field of any other type throws this IllegalArgumentException.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/io/TFRecordWriteSchemaTransformProvider.java:146

        writeTransform = writeTransform.withNumShards(configuration.getNumShards());
      } else {
        writeTransform = writeTransform.withoutSharding();
      }
      if (Boolean.TRUE.equals(configuration.getNoSpilling())) {
        writeTransform = writeTransform.withNoSpilling();
      }
      if (configuration.getMaxNumWritersPerBundle() != null) {
        writeTransform =
            writeTransform.withMaxNumWritersPerBundle(configuration.getMaxNumWritersPerBundle());
      }

      // Obtain input schema and verify only one field and its bytes
      Schema inputSchema = input.get(INPUT).getSchema();
      int numFields = inputSchema.getFields().size();
      if (numFields != 1) {
        throw new IllegalArgumentException("Expecting exactly one field, found " + numFields);
      } else if (!inputSchema.getField(0).getType().equals(Schema.FieldType.BYTES)) {
        throw new IllegalArgumentException(
            "The input schema must have exactly one field of type byte.");
      }

      final String schemaField;
      if (inputSchema.getField(0).getName() != null) {
        schemaField = inputSchema.getField(0).getName();
      } else {
        schemaField = "record";
      }

      PCollection<Row> inputRows = input.get(INPUT);

      // Convert Beam Rows to byte arrays
      SerializableFunction<Row, byte[]> rowToBytesFn = getRowToBytesFn(schemaField);

      Schema errorSchema = ErrorHandling.errorSchema(inputSchema);
      boolean handleErrors = ErrorHandling.hasOutput(configuration.getErrorHandling());

View on GitHub (pinned to 12126d8942)

Solutions

  1. Convert the field to byte[] upstream (e.g. String.getBytes(StandardCharsets.UTF_8)) so the schema field type is BYTES.
  2. Change the upstream MapElements/SQL cast to emit TypeDescriptor.of(byte[].class) / BYTES.
  3. Use a format designed for typed data (Parquet/Avro transforms) if you don't want manual serialization.

Example fix

// before
PCollection<String> lines = ...;
lines.apply(TFRecordWriteSchemaTransformProvider...); // field type STRING

// after
PCollection<byte[]> bytes = lines.apply(MapElements.into(TypeDescriptor.of(byte[].class)).via(s -> s.getBytes(StandardCharsets.UTF_8)));
bytes.apply(TFRecordWriteSchemaTransformProvider...); // field type BYTES
Defensive patterns

Strategy: type-guard

Validate before calling

Schema s = pc.getSchema();
if (s.getFieldCount() == 1 && !s.getField(0).getType().equals(Schema.FieldType.BYTES)) {
  throw new IllegalArgumentException("Sole field must be BYTES, got " + s.getField(0).getType());
}

Type guard

static boolean isBytesTyped(PCollection<?> pc) {
  Schema s = pc.getSchema();
  return s.getFieldCount() == 1 && s.getField(0).getType().equals(Schema.FieldType.BYTES);
}

Try / catch

try { pc.apply(tfRecordWrite); } catch (IllegalArgumentException e) { if (e.getMessage().contains("exactly one field of type byte")) { log.error("Field {} is {}; convert to byte[] first", pc.getSchema().getField(0).getName(), pc.getSchema().getField(0).getType()); } throw e; }

Prevention

When it happens

Trigger: Applying the TFRecord write SchemaTransform to a PCollection whose sole schema field is a non-BYTES type (STRING, INT64, ROW, etc.).

Common situations: Writing strings or numeric values directly without converting to byte[]; a prior transform leaving the field typed as VARCHAR/STRING; assuming the provider auto-serializes.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/c601e7cebf53fc6f. Report an issue: GitHub.