apache/beam · error · InvalidConfigurationException

Configuration schema provided does not match expected

Error message

Configuration schema provided does not match expected

What it means

PubsubSchemaIOProvider.validateConfigurationSchema() checks that the configuration Row passed to from() has exactly the schema returned by configurationSchema(). Any mismatch in field names, types, or order throws an InvalidConfigurationException.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubSchemaIOProvider.java:178

              + "CREATE TABLE for Pubsub topic must not be null");
    }
    if (!PubsubSchemaIO.fieldPresent(schema, TIMESTAMP_FIELD, FieldType.DATETIME)) {
      throw new InvalidSchemaException(
          "Unsupported schema specified for Pubsub source in CREATE TABLE."
              + "CREATE TABLE for Pubsub topic must include at least 'event_timestamp' field of "
              + "type 'TIMESTAMP'");
    }
  }

  private void validateDlq(@Nullable String deadLetterQueue) {
    if (deadLetterQueue != null && deadLetterQueue.isEmpty()) {
      throw new InvalidConfigurationException("Dead letter queue topic name is not specified");
    }
  }

  private void validateConfigurationSchema(Row configuration) {
    if (!configuration.getSchema().equals(configurationSchema())) {
      throw new InvalidConfigurationException(
          "Configuration schema provided does not match expected");
    }
  }

  /** An abstraction to create schema aware IOs. */
  private static class PubsubSchemaIO implements SchemaIO, Serializable {
    protected final Schema dataSchema;
    protected final String location;
    protected final boolean useFlatSchema;
    protected final Config config;

    private PubsubSchemaIO(String location, Row config, Schema dataSchema) {
      this.dataSchema = dataSchema;
      this.location = location;
      this.useFlatSchema = !shouldUseNestedSchema(dataSchema);
      this.config =
          new AutoValueSchema().fromRowFunction(TypeDescriptor.of(Config.class)).apply(config);
    }

View on GitHub (pinned to 12126d8942)

Solutions

Build the configuration Row using the provider's own configurationSchema() (or the SchemaTransform providers' configuration Row builders) instead of hand-rolling a Row.
Verify the Row's schema equals PubsubSchemaIOProvider.configurationSchema() before calling from().
Upgrade/refresh code that hard-codes the config Row to match the SDK version's expected fields.

Example fix

// before
Row config = Row.withSchema(myAdHocSchema).addValues(topic).build();
// after
Row config = Row.withSchema(new PubsubSchemaIOProvider().configurationSchema()).addValues(topic, dlq, ...).build();

Defensive patterns

Strategy: validation

Validate before calling

if (!configRow.getSchema().equals(new PubsubSchemaIOProvider().configurationSchema())) { throw new IllegalArgumentException("Configuration Row schema does not match provider's configurationSchema()"); }

Try / catch

try { ioProvider.from(configRow); } catch (InvalidConfigurationException e) { if (e.getMessage().contains("does not match expected")) { /* rebuild Row using configurationSchema() */ } throw e; }

Prevention

Never hand-build the configuration Row; construct it from the provider's configurationSchema().
Re-check config Row construction after Beam upgrades, since the expected schema may change.
Add an equality check against configurationSchema() in pipeline setup tests.

When it happens

Trigger: Calling PubsubSchemaIOProvider.from(configRow) with a hand-built or stale Row whose schema differs from the provider's expected configurationSchema() — e.g. after upgrading Beam where the config schema gained/renamed fields.

Common situations: Manually constructing the configuration Row instead of using the provider's schema/builders; cached config Rows built against an older Beam version; fields added or reordered in a newer SDK release.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/77da82eb5af6083a. Report an issue: GitHub.