Both numStorageWriteApiStreams and auto-sharding options…

Error message

Both numStorageWriteApiStreams and auto-sharding options are set. Will default to auto-sharding. To set a fixed number of streams, do not enable auto-sharding.

What it means

When writing via STORAGE_WRITE_API, BigQueryIO lets you either fix the number of write streams (numStorageWriteApiStreams) or let Beam auto-shard streams dynamically. Both cannot apply at once; auto-sharding wins, so the sink warns that the fixed stream count is being disregarded.

Solutions

Disable auto-sharding (remove withAutoSharding(true)) to honor the fixed stream count
Remove withStorageApiNumStreams(...) and keep auto-sharding for elastic throughput
Keep auto-sharding and accept the warning; the fixed value is simply ignored

Example fix

// before
BigQueryIO.writeTableRows().to(table).withMethod(Method.STORAGE_WRITE_API).withAutoSharding().withStorageApiNumStreams(10)
// after
BigQueryIO.writeTableRows().to(table).withMethod(Method.STORAGE_WRITE_API).withStorageApiNumStreams(10) // auto-sharding removed

Defensive patterns

Strategy: validation

Validate before calling

boolean conflict = autoSharding && storageApiNumStreams > 0 && method == Method.STORAGE_WRITE_API; if (conflict) { /* choose one */ }

Prevention

Choose one sharding strategy per write: auto-sharding OR fixed streams
Remove stale numStorageWriteApiStreams tuning after enabling auto-sharding
Document the chosen streaming-write config in pipeline READMEs

When it happens

Trigger: Calling withStorageApiNumStreams(n) (or setting the numStorageWriteApiStreams option) together with withAutoSharding(true) (triggering the default auto-sharding) while method is STORAGE_WRITE_API.

Common situations: Copy-pasted streaming write configs where a fixed stream count was tuned earlier and auto-sharding was later enabled for elasticity, leaving the fixed count stale.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f2d3a3eb1b10499e. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java:3881

              "Triggering frequency can be specified only when writing via FILE_LOADS or STORAGE_WRITE_API, but the method was %s.",
              method);
        }
        if (method != Method.FILE_LOADS) {
          checkArgument(
              getNumFileShards() == 0,
              "Number of file shards can be specified only when writing via FILE_LOADS, but the method was %s.",
              method);
        }
        if (method == Method.STORAGE_API_AT_LEAST_ONCE
            && getStorageApiTriggeringFrequency(bqOptions) != null) {
          LOG.warn(
              "Storage API triggering frequency option will be ignored is it can only be specified only "
                  + "when writing via STORAGE_WRITE_API, but the method was {}.",
              method);
        }
        if (getAutoSharding()) {
          if (method == Method.STORAGE_WRITE_API && getStorageApiNumStreams(bqOptions) > 0) {
            LOG.warn(
                "Both numStorageWriteApiStreams and auto-sharding options are set. Will default to auto-sharding."
                    + " To set a fixed number of streams, do not enable auto-sharding.");
          } else if (method == Method.FILE_LOADS && getNumFileShards() > 0) {
            LOG.warn(
                "Both numFileShards and auto-sharding options are set. Will default to auto-sharding."
                    + " To set a fixed number of file shards, do not enable auto-sharding.");
          } else if (method == Method.STORAGE_API_AT_LEAST_ONCE) {
            LOG.warn(
                "The setting of auto-sharding is ignored. It is only supported when writing an"
                    + " unbounded PCollection via FILE_LOADS, STREAMING_INSERTS or"
                    + " STORAGE_WRITE_API, but the method was {}.",
                method);
          }
        }
      } else { // PCollection is bounded
        checkArgument(
            getTriggeringFrequency() == null,
            "Triggering frequency is only applicable to an unbounded PCollection.");

View on GitHub (pinned to 12126d8942)