{"record":{"id":"7f0b695657521e23","repo":"apache/beam","slug":"both-numfileshards-and-auto-sharding-options-are-set-will","errorCode":null,"errorMessage":"Both numFileShards and auto-sharding options are set. Will default to auto-sharding. To set a fixed number of file shards, do not enable auto-sharding.","messagePattern":"Both numFileShards and auto-sharding options are set\\. Will default to auto-sharding\\. To set a fixed number of file shards, do not enable auto-sharding\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java","lineNumber":3885,"sourceCode":"          checkArgument(\n              getNumFileShards() == 0,\n              \"Number of file shards can be specified only when writing via FILE_LOADS, but the method was %s.\",\n              method);\n        }\n        if (method == Method.STORAGE_API_AT_LEAST_ONCE\n            && getStorageApiTriggeringFrequency(bqOptions) != null) {\n          LOG.warn(\n              \"Storage API triggering frequency option will be ignored is it can only be specified only \"\n                  + \"when writing via STORAGE_WRITE_API, but the method was {}.\",\n              method);\n        }\n        if (getAutoSharding()) {\n          if (method == Method.STORAGE_WRITE_API && getStorageApiNumStreams(bqOptions) > 0) {\n            LOG.warn(\n                \"Both numStorageWriteApiStreams and auto-sharding options are set. Will default to auto-sharding.\"\n                    + \" To set a fixed number of streams, do not enable auto-sharding.\");\n          } else if (method == Method.FILE_LOADS && getNumFileShards() > 0) {\n            LOG.warn(\n                \"Both numFileShards and auto-sharding options are set. Will default to auto-sharding.\"\n                    + \" To set a fixed number of file shards, do not enable auto-sharding.\");\n          } else if (method == Method.STORAGE_API_AT_LEAST_ONCE) {\n            LOG.warn(\n                \"The setting of auto-sharding is ignored. It is only supported when writing an\"\n                    + \" unbounded PCollection via FILE_LOADS, STREAMING_INSERTS or\"\n                    + \" STORAGE_WRITE_API, but the method was {}.\",\n                method);\n          }\n        }\n      } else { // PCollection is bounded\n        checkArgument(\n            getTriggeringFrequency() == null,\n            \"Triggering frequency is only applicable to an unbounded PCollection.\");\n        checkArgument(\n            !getAutoSharding(), \"Auto-sharding is only applicable to an unbounded PCollection.\");\n        checkArgument(\n            getNumFileShards() == 0,","sourceCodeStart":3867,"sourceCodeEnd":3903,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java#L3867-L3903","documentation":"For FILE_LOADS writes, BigQueryIO can use a fixed number of file shards or auto-shard the output files. Setting both is contradictory; auto-sharding takes precedence and the fixed numFileShards value is ignored, so the sink logs this warning.","triggerScenarios":"Calling withAutoSharding(true) while withNumFileShards(n) is also set and the write method is FILE_LOADS.","commonSituations":"Batch pipelines that tuned numFileShards for load performance, then enabled auto-sharding during a refactor to support unbounded input, leaving both settings active.","solutions":["Remove withNumFileShards(...) and rely on auto-sharding","Disable auto-sharding to use the explicit shard count","Accept the warning knowing numFileShards is ignored"],"exampleFix":"// before\nBigQueryIO.writeTableRows().to(table).withMethod(Method.FILE_LOADS).withAutoSharding().withNumFileShards(100)\n// after\nBigQueryIO.writeTableRows().to(table).withMethod(Method.FILE_LOADS).withNumFileShards(100) // auto-sharding removed","handlingStrategy":"validation","validationCode":"boolean conflict = autoSharding && numFileShards > 0 && method == Method.FILE_LOADS; if (conflict) { /* pick auto-sharding or fixed shards */ }","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Do not mix withNumFileShards and withAutoSharding in the same sink builder","Revisit shard settings when converting batch pipelines to streaming","Grep pipelines for both setters during config reviews"],"tags":["java","apache-beam","bigquery","configuration"],"backgroundTag":"conflicting-config-options","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}