apache/beam · error · IllegalArgumentException

Failed to decode Schema due to an error decoding Field proto

Error message

Failed to decode Schema due to an error decoding Field proto:

+protoField

What it means

schemaFromProto wraps any failure while translating a single Field proto into an IllegalArgumentException with the message 'Failed to decode Schema due to an error decoding Field proto' plus the proto's text form. The original exception (e.g. bad logical type, unknown atomic type) is attached as the cause.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/SchemaTranslation.java:310

        builder.setAtomicType(SchemaApi.AtomicType.BOOLEAN);
        break;
      case BYTES:
        builder.setAtomicType(SchemaApi.AtomicType.BYTES);
        break;
    }
    builder.setNullable(fieldType.getNullable());
    return builder.build();
  }

  public static Schema schemaFromProto(SchemaApi.Schema protoSchema) {
    Schema.Builder builder = Schema.builder();
    Map<String, Integer> encodingLocationMap = Maps.newHashMap();
    for (SchemaApi.Field protoField : protoSchema.getFieldsList()) {
      Field field;
      try {
        field = fieldFromProto(protoField);
      } catch (Exception e) {
        throw new IllegalArgumentException(
            "Failed to decode Schema due to an error decoding Field proto:\n\n" + protoField, e);
      }
      builder.addField(field);
      encodingLocationMap.put(protoField.getName(), protoField.getEncodingPosition());
    }
    builder.setOptions(optionsFromProto(protoSchema.getOptionsList()));
    Schema schema = builder.build();

    Preconditions.checkState(encodingLocationMap.size() == schema.getFieldCount());
    long distinctEncodingPositions = encodingLocationMap.values().stream().distinct().count();
    Preconditions.checkState(distinctEncodingPositions <= schema.getFieldCount());
    if (distinctEncodingPositions < schema.getFieldCount() && schema.getFieldCount() > 0) {
      // This means that encoding positions were not specified in the proto. Generally, we don't
      // expect this to happen,
      // but if it does happen, we expect none to be specified - in which case the should all be
      // zero.
      Preconditions.checkState(distinctEncodingPositions == 1);
    } else if (protoSchema.getEncodingPositionsSet()) {

View on GitHub (pinned to 12126d8942)

Solutions

1. Inspect the cause exception printed below the message; it identifies the exact failing field type.
2. Ensure the SDK version decoding the data is >= the version that encoded it (or compatible schema capabilities).
3. Check the field's type_name/urn in the proto; replace unsupported or hand-edited field definitions.
4. Catch the IllegalArgumentException when decoding untrusted/older payloads and treat the record as unparseable.

Example fix

// before
Schema s = SchemaTranslation.schemaFromProto(suspiciousProto);
// after
try { Schema s = SchemaTranslation.schemaFromProto(proto); }
catch (IllegalArgumentException e) { LOG.error("Bad field: {}", e.getMessage(), e.getCause()); }

protoSchema.getFieldsList().forEach(f -> {
  if (f.getName().isEmpty() || !f.hasFieldType())
    throw new IllegalArgumentException("Malformed field: " + f);
});

try {
  Schema s = SchemaTranslation.schemaFromProto(proto);
} catch (IllegalArgumentException e) {
  LOG.error("Field decode failed: {} cause: {}", e.getMessage(), e.getCause());
  throw new DataFormatException("Undecodable schema", e);
}

When it happens

Trigger: Decoding a serialized Schema (schemaFromProto / fromProto on rows) where at least one Field proto is malformed: unknown type name, unsupported options, or nested field translation failure (see errors 303-309).

Common situations: Cross-version Beam data exchange where a newer field type was serialized and an older SDK cannot decode it; manually constructed/edited schema protos; corrupted pipeline state.

Understand the failure class

Background: "cannot parse invalid wire-format data", "cannot unmarshal", "failed unmarshalling": protobuf unmarshal errors explained — this error's family across 10 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Encountered UNSPECIFIED AtomicType
• Cannot provide a coder for a Beam Row. Please provide a sche
• Could not decode bytes as message
• Could not encode message as bytes
• Encountered unknown AtomicType: +protoFieldType.getAtomicTyp

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/3565fd43a73daf49. Report an issue: GitHub.