apache/beam · info

Both withDataSourceConfiguration and…

Error message

Both withDataSourceConfiguration and withDataSourceProviderFn are provided. dataSourceProviderFn will override dataSourceConfiguration.

What it means

A JdbcIO.ReadRows builder warning: the user has called both withDataSourceConfiguration and a custom withDataSourceProviderFn. The custom provider function will take precedence and the data source configuration will be ignored. The builder does not fail — it logs this warning and proceeds.

Solutions

1. Remove the withDataSourceConfiguration(...) call if you intend to use your custom provider function
2. Or remove withDataSourceProviderFn and rely solely on withDataSourceConfiguration
3. Verify at runtime which DataSource is actually used (the custom fn wins) by inspecting connection properties

Example fix

// before
JdbcIO.<T>readRows()
  .withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create(driver, url))
  .withDataSourceProviderFn(myHikariFn)
// after
JdbcIO.<T>readRows()
  .withDataSourceProviderFn(myHikariFn)

// guard in your pipeline-construction helper
if (config != null && providerFn != null) {
  LOG.warn("Pick one data source mechanism; providerFn will override config");
  config = null;
}

When it happens

Trigger: Calling withDataSourceProviderFn(fn) on a ReadRows whose dataSourceProviderFn is already set to something other than the default DataSourceProviderFromDataSourceConfiguration wrapper — i.e., a genuinely custom fn plus an explicit DataSourceConfiguration.

Common situations: Copy-pasted fluent builder chains where withDataSourceConfiguration(...) was left in place before adding a custom DataSource provider (e.g., HikariCP-backed fn), migrating configuration-based setup to a pooled DataSource without removing the old call.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

• Either Write Statement or Table must be set.
• If JDBC type is not specified, then Driver Class Name and…
• JDBC type must be one of
• JDBC URL cannot be blank
• One of JDBC Driver class name or JDBC type must be…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/86b7435c4a9f70c5. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcIO.java:881

      abstract Builder setOutputParallelization(boolean outputParallelization);

      abstract Builder setDisableAutoCommit(boolean disableAutoCommit);

      abstract Builder setSchema(@Nullable Schema schema);

      abstract ReadRows build();
    }

    public ReadRows withDataSourceConfiguration(DataSourceConfiguration config) {
      return withDataSourceProviderFn(new DataSourceProviderFromDataSourceConfiguration(config));
    }

    public ReadRows withDataSourceProviderFn(
        SerializableFunction<Void, DataSource> dataSourceProviderFn) {
      if (getDataSourceProviderFn() != null
          && !(getDataSourceProviderFn()
              instanceof DataSourceProviderFromDataSourceConfiguration)) {
        LOG.warn(
            "Both withDataSourceConfiguration and withDataSourceProviderFn are provided. dataSourceProviderFn will override dataSourceConfiguration.");
      }
      return toBuilder().setDataSourceProviderFn(dataSourceProviderFn).build();
    }

    public ReadRows withQuery(String query) {
      checkArgument(query != null, "query can not be null");
      return withQuery(ValueProvider.StaticValueProvider.of(query));
    }

    public ReadRows withQuery(ValueProvider<String> query) {
      checkArgument(query != null, "query can not be null");
      return toBuilder().setQuery(query).build();
    }

    public ReadRows withStatementPreparator(StatementPreparator statementPreparator) {
      checkArgument(statementPreparator != null, "statementPreparator can not be null");
      return toBuilder().setStatementPreparator(statementPreparator).build();

View on GitHub (pinned to 12126d8942)