apache/beam · warning
Storage API triggering frequency option will be ignored is…
Error message
Storage API triggering frequency option will be ignored is it can only be specified only when writing via STORAGE_WRITE_API, but the method was {}. What it means
BigQueryIO supports withStorageApiTriggeringFrequencyOnly... this option (set via withTriggeringFrequency in the storage-write path) only controls flush cadence when the write method is STORAGE_WRITE_API. When a different method is selected, the triggering frequency has no effect, so the sink logs this warning instead of silently ignoring it.
Solutions
- Switch the write method to Method.STORAGE_WRITE_API if triggering frequency is required
- Remove the withTriggeringFrequency(...) call when using STORAGE_API_AT_LEAST_ONCE or other methods
- Note that STORAGE_API_AT_LEAST_ONCE flushes per bundle; rely on auto-sharding/streams for throughput instead
Example fix
// before BigQueryIO.writeTableRows().to(table).withMethod(Method.STORAGE_API_AT_LEAST_ONCE).withTriggeringFrequency(Duration.standardSeconds(10)) // after BigQueryIO.writeTableRows().to(table).withMethod(Method.STORAGE_API_AT_LEAST_ONCE) // triggering frequency removed
Defensive patterns
Strategy: validation
Validate before calling
if (method != Method.STORAGE_WRITE_API && triggeringFrequency != null) { log.warn("triggeringFrequency ignored for method " + method); } Prevention
- Only set withTriggeringFrequency when method == STORAGE_WRITE_API
- Keep sink option builders per method to avoid cross-method option leakage
- Treat BigQueryIO startup warnings as config-review signals in CI logs
When it happens
Trigger: Using write().withMethod(STORAGE_API_AT_LEAST_ONCE) (or another non-STORAGE_WRITE_API method) while getStorageApiTriggeringFrequency resolves to a non-null value — typically because withTriggeringFrequency(Duration) was set for a streaming insert path.
Common situations: Migrating a pipeline from legacy STREAMING_INSERTS with a triggering frequency to the Storage API at-least-once mode and leaving the frequency setter in place.
Understand the failure class
Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.
Related errors
- Both numFileShards and auto-sharding options are set. Will…
- Both numStorageWriteApiStreams and auto-sharding options…
- Setting a number of Storage API streams is only supported…
- The setting of auto-sharding is ignored. It is only…
- A function must be provided to convert the input type into…
AI-assisted analysis of apache/beam@12126d8942 (2026-09-13).
Data as JSON: /api/errors/8a65f6236a04d75a.
Report an issue: GitHub.
Appendix: source
Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java:3874
checkArgument(
triggeringFrequency != null,
"When writing an unbounded PCollection via FILE_LOADS or STORAGE_WRITE_API, "
+ "triggering frequency must be specified");
} else {
checkArgument(
getTriggeringFrequency() == null,
"Triggering frequency can be specified only when writing via FILE_LOADS or STORAGE_WRITE_API, but the method was %s.",
method);
}
if (method != Method.FILE_LOADS) {
checkArgument(
getNumFileShards() == 0,
"Number of file shards can be specified only when writing via FILE_LOADS, but the method was %s.",
method);
}
if (method == Method.STORAGE_API_AT_LEAST_ONCE
&& getStorageApiTriggeringFrequency(bqOptions) != null) {
LOG.warn(
"Storage API triggering frequency option will be ignored is it can only be specified only "
+ "when writing via STORAGE_WRITE_API, but the method was {}.",
method);
}
if (getAutoSharding()) {
if (method == Method.STORAGE_WRITE_API && getStorageApiNumStreams(bqOptions) > 0) {
LOG.warn(
"Both numStorageWriteApiStreams and auto-sharding options are set. Will default to auto-sharding."
+ " To set a fixed number of streams, do not enable auto-sharding.");
} else if (method == Method.FILE_LOADS && getNumFileShards() > 0) {
LOG.warn(
"Both numFileShards and auto-sharding options are set. Will default to auto-sharding."
+ " To set a fixed number of file shards, do not enable auto-sharding.");
} else if (method == Method.STORAGE_API_AT_LEAST_ONCE) {
LOG.warn(
"The setting of auto-sharding is ignored. It is only supported when writing an"
+ " unbounded PCollection via FILE_LOADS, STREAMING_INSERTS or"
+ " STORAGE_WRITE_API, but the method was {}.",View on GitHub (pinned to 12126d8942)