What causes "Incorrectly sized byte array." in apache/beam?

Pad or truncate the byte[] to the exact fixed size before conversion (e.g. Arrays.copyOf(byteArray, fixedSize))

apache/beam · error · IllegalArgumentException

Incorrectly sized byte array.

Error message

Incorrectly sized byte array.

What it means

Thrown by AvroUtils.genericFromBeamField when converting a Beam FixedBytes logical-type field to an Avro fixed schema: the byte[] value's length does not equal the declared fixed size (FixedBytesField.getSize()). Avro 'fixed' values must be exactly the declared size, so short or long arrays are rejected.

Solutions

Pad or truncate the byte[] to the exact fixed size before conversion (e.g. Arrays.copyOf(byteArray, fixedSize))
Verify the producer writes arrays of exactly FixedBytesField.getSize() bytes
Update the Beam FixedBytes.of(n) size (and Avro fixed size) to match the actual data length
Switch to the VariableBytes logical type if lengths genuinely vary

Example fix

// before
byte[] value = Arrays.copyOfRange(raw, 0, 5); // fixed size is 8
row = row.withValue("digest", value);
// after
byte[] value = Arrays.copyOf(raw, 8); // pad to fixed size 8
row = row.withValue("digest", value);

Defensive patterns

Strategy: validation

Validate before calling

boolean hasFixedSize(byte[] value, Schema.FieldType fieldType) {
  FixedBytesField f = FixedBytesField.fromBeamFieldType(fieldType);
  return f != null && value.length == f.getSize();
}

Try / catch

try {
  GenericRecord record = AvroUtils.toGenericRecord(row, avroSchema);
} catch (IllegalArgumentException e) {
  if ("Incorrectly sized byte array.".equals(e.getMessage())) {
    throw new IllegalStateException("Pad/trim byte arrays to the declared fixed size before conversion", e);
  } else { throw e; }
}

Prevention

Validate byte[] lengths at Row-construction time, not at Avro conversion time
Keep FixedBytes.of(n) sizes and Avro fixed(n) sizes defined from one shared constant
If lengths genuinely vary, use VariableBytes instead of FixedBytes

When it happens

Trigger: Converting a Beam Row whose field is a FixedBytes logical type to a GenericRecord where value is a byte[] with length != fixed size — e.g. writing a 5-byte array into a fixed(8) Avro field, or padding/truncation was skipped upstream.

Common situations: Reading fixed-size values from a database or file into byte[] and forgetting to pad to the declared length; changing the fixed size in the Beam schema (FixedBytes.of(n)) without updating producer code; deserializing data produced under an older, different fixed size.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/c415569eddbb3383. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/avro/src/main/java/org/apache/beam/sdk/extensions/avro/schemas/utils/AvroUtils.java:1342

        } else if (typeWithNullability.type.getType() == org.apache.avro.Schema.Type.LONG) {
          ReadableInstant instant = (ReadableInstant) value;
          return (long) instant.getMillis();
        } else {
          throw new IllegalArgumentException(
              "Can't represent " + fieldType + " as " + typeWithNullability.type.getType());
        }

      case BYTES:
        return ByteBuffer.wrap((byte[]) value);

      case LOGICAL_TYPE:
        String identifier = checkNotNull(fieldType.getLogicalType()).getIdentifier();
        if (FixedBytes.IDENTIFIER.equals(identifier)) {
          FixedBytesField fixedBytesField =
              checkNotNull(FixedBytesField.fromBeamFieldType(fieldType));
          byte[] byteArray = (byte[]) value;
          if (byteArray.length != fixedBytesField.getSize()) {
            throw new IllegalArgumentException("Incorrectly sized byte array.");
          }
          return NullnessCheckerWorkarounds.createFixed(
              null, (byte[]) value, typeWithNullability.type);
        } else if (VariableBytes.IDENTIFIER.equals(identifier)) {
          return NullnessCheckerWorkarounds.createFixed(
              null, (byte[]) value, typeWithNullability.type);
        } else if (FixedString.IDENTIFIER.equals(identifier)
            || "CHAR".equals(identifier)
            || "NCHAR".equals(identifier)) {
          return new Utf8((String) value);
        } else if (VariableString.IDENTIFIER.equals(identifier)
            || "NVARCHAR".equals(identifier)
            || "VARCHAR".equals(identifier)
            || "LONGNVARCHAR".equals(identifier)
            || "LONGVARCHAR".equals(identifier)) {
          return new Utf8((String) value);
        } else if (EnumerationType.IDENTIFIER.equals(identifier)) {
          EnumerationType enumerationType = fieldType.getLogicalType(EnumerationType.class);

View on GitHub (pinned to 12126d8942)