apache/beam · error · BeamSchemaInferenceException

Failed to infer Beam schema

Error message

Failed to infer Beam schema

What it means

JdbcIO.ReadRows infers a Beam Schema by executing the query and converting ResultSetMetaData via SchemaUtil.toBeamSchema. A SQLException during statement preparation, execution, or metadata retrieval is surfaced as BeamSchemaInferenceException with this message.

Source

Thrown at sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcIO.java:981

                  .withDisableAutoCommit(getDisableAutoCommit())
                  .withOutputParallelization(getOutputParallelization())
                  .withStatementPreparator(checkStateNotNull(getStatementPreparator())));
      rows.setRowSchema(schema);
      return rows;
    }

    // Spotbugs seems to not understand the multi-statement try-with-resources
    @SuppressFBWarnings("OBL_UNSATISFIED_OBLIGATION")
    public static Schema inferBeamSchema(DataSource ds, String query) {
      try (Connection conn = ds.getConnection();
          PreparedStatement statement =
              conn.prepareStatement(
                  query, ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY)) {
        ResultSetMetaData metadata =
            checkStateNotNull(statement.getMetaData(), "could not get statement metadata");
        return SchemaUtil.toBeamSchema(metadata);
      } catch (SQLException e) {
        throw new BeamSchemaInferenceException("Failed to infer Beam schema", e);
      }
    }

    @Override
    public void populateDisplayData(DisplayData.Builder builder) {
      super.populateDisplayData(builder);
      builder.add(DisplayData.item("query", getQuery()));
      if (getDataSourceProviderFn() instanceof HasDisplayData) {
        ((HasDisplayData) getDataSourceProviderFn()).populateDisplayData(builder);
      }
    }
  }

  /** Implementation of {@link #read}. */
  @AutoValue
  public abstract static class Read<T> extends PTransform<PBegin, PCollection<T>> {

    @Pure

View on GitHub (pinned to 12126d8942)

Solutions

1. Run the query directly against the database (psql/sqlplus/jdbc client) to confirm it is valid and authorized
2. Verify connection configuration (URL, user, password, driver jar in the classpath / --jars)
3. Catch BeamSchemaInferenceException and fall back to an explicitly provided schema via withRowMapper/withSchema if inference is unsupported for your driver

Example fix

// before
JdbcIO.<Void>readRows().withQuery("SELCT * FROM users").withDataSourceConfiguration(config);
// after
JdbcIO.<Void>readRows().withQuery("SELECT * FROM users").withDataSourceConfiguration(config);

// preflight: run the query on a plain connection before building the pipeline
try (Connection c = config.buildOrGetConnectionFactory().getConnection();
     PreparedStatement ps = c.prepareStatement(query, ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY)) {
  checkState(ps.getMetaData() != null, "driver returns no metadata for: " + query);
}

try {
  pipeline.apply(JdbcIO.<Void>readRows().withQuery(query).withDataSourceConfiguration(cfg));
} catch (BeamSchemaInferenceException e) {
  // fall back to explicit schema or fix query/credentials
}

When it happens

Trigger: Running JdbcIO.readRows() where the prepared statement's metadata cannot be fetched — invalid SQL, nonexistent table/columns, insufficient privileges, unreachable database, or a driver that does not return metadata for the prepared statement.

Common situations: Typo in query or table name; missing driver dependency on the classpath; database credentials/network problems; DB user lacking metadata read privileges; statement.getMetaData() returning null for exotic drivers.

Understand the failure class

Background: "query failed", "%w: SQL error" — wrapped database query errors in Go libraries explained — this error's family across 3 libraries.

Related errors

• Beam JDBC connection has not been initialized
• Unable to parse statement
• Error while determining columns from table: ${getTable()}
• Unable to create prepared statement for type: ${type}
• Unable run pipeline with EMPTY disposition.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/6349853da6b39ed6. Report an issue: GitHub.