Both numFileShards and auto-sharding options are set. Will…

Error message

Both numFileShards and auto-sharding options are set. Will default to auto-sharding. To set a fixed number of file shards, do not enable auto-sharding.

What it means

For FILE_LOADS writes, BigQueryIO can use a fixed number of file shards or auto-shard the output files. Setting both is contradictory; auto-sharding takes precedence and the fixed numFileShards value is ignored, so the sink logs this warning.

Solutions

Remove withNumFileShards(...) and rely on auto-sharding
Disable auto-sharding to use the explicit shard count
Accept the warning knowing numFileShards is ignored

Example fix

// before
BigQueryIO.writeTableRows().to(table).withMethod(Method.FILE_LOADS).withAutoSharding().withNumFileShards(100)
// after
BigQueryIO.writeTableRows().to(table).withMethod(Method.FILE_LOADS).withNumFileShards(100) // auto-sharding removed

Defensive patterns

Strategy: validation

Validate before calling

boolean conflict = autoSharding && numFileShards > 0 && method == Method.FILE_LOADS; if (conflict) { /* pick auto-sharding or fixed shards */ }

Prevention

Do not mix withNumFileShards and withAutoSharding in the same sink builder
Revisit shard settings when converting batch pipelines to streaming
Grep pipelines for both setters during config reviews

When it happens

Trigger: Calling withAutoSharding(true) while withNumFileShards(n) is also set and the write method is FILE_LOADS.

Common situations: Batch pipelines that tuned numFileShards for load performance, then enabled auto-sharding during a refactor to support unbounded input, leaving both settings active.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7f0b695657521e23. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java:3885

          checkArgument(
              getNumFileShards() == 0,
              "Number of file shards can be specified only when writing via FILE_LOADS, but the method was %s.",
              method);
        }
        if (method == Method.STORAGE_API_AT_LEAST_ONCE
            && getStorageApiTriggeringFrequency(bqOptions) != null) {
          LOG.warn(
              "Storage API triggering frequency option will be ignored is it can only be specified only "
                  + "when writing via STORAGE_WRITE_API, but the method was {}.",
              method);
        }
        if (getAutoSharding()) {
          if (method == Method.STORAGE_WRITE_API && getStorageApiNumStreams(bqOptions) > 0) {
            LOG.warn(
                "Both numStorageWriteApiStreams and auto-sharding options are set. Will default to auto-sharding."
                    + " To set a fixed number of streams, do not enable auto-sharding.");
          } else if (method == Method.FILE_LOADS && getNumFileShards() > 0) {
            LOG.warn(
                "Both numFileShards and auto-sharding options are set. Will default to auto-sharding."
                    + " To set a fixed number of file shards, do not enable auto-sharding.");
          } else if (method == Method.STORAGE_API_AT_LEAST_ONCE) {
            LOG.warn(
                "The setting of auto-sharding is ignored. It is only supported when writing an"
                    + " unbounded PCollection via FILE_LOADS, STREAMING_INSERTS or"
                    + " STORAGE_WRITE_API, but the method was {}.",
                method);
          }
        }
      } else { // PCollection is bounded
        checkArgument(
            getTriggeringFrequency() == null,
            "Triggering frequency is only applicable to an unbounded PCollection.");
        checkArgument(
            !getAutoSharding(), "Auto-sharding is only applicable to an unbounded PCollection.");
        checkArgument(
            getNumFileShards() == 0,

View on GitHub (pinned to 12126d8942)