apache/beam · error · IllegalArgumentException

Given message schema

Error message

Given message schema: '%s'%ndoes not match schema inferred from thrift class.%nThrift class: '%s'%nInferred schema: '%s'

What it means

inferAndVerifySchema derives a Beam Schema from the thrift class and checks that the user-provided target schema is assignable to it. When the provided schema does not match the schema inferred from the thrift class, an IllegalArgumentException with this formatted message (showing both schemas and the class) is thrown.

Solutions

1. Compare the 'Inferred schema' in the message with your provided schema and align field names, types, and nullability
2. Regenerate thrift classes after IDL changes and re-derive the schema instead of maintaining it by hand
3. Use the schema inferred from the thrift class (ThriftSchema.provider().schemaFor(TypeDescriptor.of(cls))) rather than a hand-built one

Example fix

// before
Schema mySchema = Schema.of(Field.of("name", FieldType.STRING), Field.of("age", FieldType.INT32));
thriftIO.withSchema(mySchema); // mismatches inferred schema
// after
Schema inferred = ThriftSchema.provider()
    .schemaFor(TypeDescriptor.of(ThriftRecord.class));
thriftIO.withSchema(inferred);

Schema inferred = ThriftSchema.provider().schemaFor(TypeDescriptor.of(ThriftRecord.class)); if (!inferred.equals(provided)) { throw new IllegalArgumentException("provided schema differs from inferred thrift schema"); }

try { thriftIO.withSchema(mySchema); } catch (IllegalArgumentException e) { schema = ThriftSchema.provider().schemaFor(TypeDescriptor.of(ThriftRecord.class)); }

When it happens

Trigger: Calling ThriftIO write/read with an explicitly supplied Beam Schema whose field names, types, nullability, or ordering don't match what ThriftSchema infers from the configured thrift class.

Common situations: Hand-writing a Beam Schema for a thrift type instead of deriving it; thrift IDL updated (field added/renamed) but the supplied schema not updated; nullable/optional mismatch between thrift requirement and Beam field nullability.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Could not infer beam type for thrift field
• Expected thrift class but got
• Given message schema
• Provided length is bigger than max length
• Provided size is bigger than max size

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/87b2edd63374e5f3. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/thrift/src/main/java/org/apache/beam/sdk/io/thrift/ThriftPayloadSerializerProvider.java:72

  }

  private static TProtocolFactory getProtocolFactory(Map<String, Object> tableParams) {
    String thriftFactoryClassName =
        checkArgumentNotNull(tableParams.get("thriftProtocolFactoryClass")).toString();
    try {
      Class<?> thriftClass = Class.forName(thriftFactoryClassName);
      return thriftClass.asSubclass(TProtocolFactory.class).getDeclaredConstructor().newInstance();
    } catch (ReflectiveOperationException e) {
      throw new IllegalArgumentException(
          "Incorrect thrift protocol factory class provided: " + thriftFactoryClassName, e);
    }
  }

  private static void inferAndVerifySchema(Class<?> thriftClass, Schema requiredSchema) {
    TypeDescriptor<?> typeDescriptor = TypeDescriptor.of(thriftClass);
    Schema schema = checkArgumentNotNull(ThriftSchema.provider().schemaFor(typeDescriptor));
    if (!schema.assignableTo(requiredSchema)) {
      throw new IllegalArgumentException(
          String.format(
              "Given message schema: '%s'%n"
                  + "does not match schema inferred from thrift class.%n"
                  + "Thrift class: '%s'%n"
                  + "Inferred schema: '%s'",
              requiredSchema, thriftClass.getName(), schema));
    }
  }

  /** A helper needed to fix the type `T` of thriftClass to satisfy RowMessages constraints. */
  private static <T extends TBase> PayloadSerializer getPayloadSerializer(
      Schema schema, TProtocolFactory protocolFactory, Class<T> thriftClass) {
    Coder<T> coder = ThriftCoder.of(thriftClass, protocolFactory);
    TypeDescriptor<T> descriptor = TypeDescriptor.of(thriftClass);
    SimpleFunction<byte[], Row> toRowFn =
        RowMessages.bytesToRowFn(ThriftSchema.provider(), descriptor, coder);
    return PayloadSerializer.of(
        RowMessages.rowToBytesFn(ThriftSchema.provider(), descriptor, coder),

View on GitHub (pinned to 12126d8942)