apache/beam · error · IllegalArgumentException

A dataSourceConfiguration or dataSourceProviderFn has…

Error message

A dataSourceConfiguration or dataSourceProviderFn has already been provided, and does not need to be provided again.

What it means

JdbcIO.ReadAll allows the data source to be supplied either as a DataSourceConfiguration or as a SerializableFunction provider fn, but only once. Calling withDataSourceProviderFn when one is already set (including implicitly via withDataSourceConfiguration) throws IllegalArgumentException.

Solutions

1. Remove the redundant withDataSourceConfiguration/withDataSourceProviderFn call, keeping only one
2. If you need a custom provider, use only withDataSourceProviderFn; if you have a config object, use only withDataSourceConfiguration
3. Guard the builder programmatically: check which setter applies to your case before building the transform

Example fix

// before
JdbcIO.<KV<String, Row>>readAll()
    .withDataSourceConfiguration(config)
    .withDataSourceProviderFn(myFn)
    .withQuery(query);
// after
JdbcIO.<KV<String, Row>>readAll()
    .withDataSourceConfiguration(config)
    .withQuery(query);

if (dataSourceConfig != null && customProviderFn != null) {
  throw new IllegalArgumentException("Provide only one of dataSourceConfiguration / dataSourceProviderFn");
}

try {
  return readAll.withDataSourceProviderFn(fn);
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("dataSourceConfiguration or dataSourceProviderFn")) {
    return readAll; // one is already set; use the existing transform
  }
  throw e;
}

When it happens

Trigger: Chaining .withDataSourceConfiguration(cfg).withDataSourceProviderFn(fn) — the first call already populates dataSourceProviderFn via DataSourceProviderFromDataSourceConfiguration, so the second setter rejects the pipeline.

Common situations: Copy-pasted builder options; merging two builder code paths that each set a data source; refactoring from DataSourceConfiguration to a custom provider without removing the original call.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

• BeamRowMapper does not have support for fields of type
• Both withDataSourceConfiguration and…
• Converting to Beam schema type is not supported
• Either Write Statement or Table must be set.
• If JDBC type is not specified, then Driver Class Name and…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d088cdabb4919986. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcIO.java:1233

      abstract Builder<ParameterT, OutputT> setFetchSize(int fetchSize);

      abstract Builder<ParameterT, OutputT> setOutputParallelization(boolean outputParallelization);

      abstract Builder<ParameterT, OutputT> setDisableAutoCommit(boolean disableAutoCommit);

      abstract ReadAll<ParameterT, OutputT> build();
    }

    public ReadAll<ParameterT, OutputT> withDataSourceConfiguration(
        DataSourceConfiguration config) {
      return withDataSourceProviderFn(new DataSourceProviderFromDataSourceConfiguration(config));
    }

    public ReadAll<ParameterT, OutputT> withDataSourceProviderFn(
        SerializableFunction<Void, DataSource> dataSourceProviderFn) {
      if (getDataSourceProviderFn() != null) {
        throw new IllegalArgumentException(
            "A dataSourceConfiguration or dataSourceProviderFn has "
                + "already been provided, and does not need to be provided again.");
      }
      return toBuilder().setDataSourceProviderFn(dataSourceProviderFn).build();
    }

    public ReadAll<ParameterT, OutputT> withQuery(String query) {
      checkArgument(query != null, "JdbcIO.readAll().withQuery(query) called with null query");
      return withQuery(ValueProvider.StaticValueProvider.of(query));
    }

    public ReadAll<ParameterT, OutputT> withQuery(ValueProvider<String> query) {
      checkArgument(query != null, "JdbcIO.readAll().withQuery(query) called with null query");
      return toBuilder().setQuery(query).build();
    }

    /**
     * Sets the {@link PreparedStatementSetter} to set the parameters of the query for each input

View on GitHub (pinned to 12126d8942)