{"record":{"id":"5b6329709604f552","repo":"apache/beam","slug":"error-handling-is-partially-supported-when-using-file-loads","errorCode":null,"errorMessage":"Error Handling is partially supported when using FILE_LOADS. Consider using STORAGE_WRITE_API or STORAGE_API_AT_LEAST_ONCE","messagePattern":"Error Handling is partially supported when using FILE_LOADS\\. Consider using STORAGE_WRITE_API or STORAGE_API_AT_LEAST_ONCE","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java","lineNumber":4227,"sourceCode":"                .withSuccessfulInsertsPropagation(getPropagateSuccessful())\n                .withDeterministicRecordIdFn(getDeterministicRecordIdFn())\n                .withKmsKey(getKmsKey());\n        return input.apply(streamingInserts);\n      } else if (method == Write.Method.FILE_LOADS) {\n        checkArgument(\n            getFailedInsertRetryPolicy() == null,\n            \"Record-insert retry policies are not supported when using BigQuery load jobs.\");\n\n        if (getUseAvroLogicalTypes()) {\n          checkArgument(\n              rowWriterFactory.getOutputType() == OutputType.AvroGenericRecord,\n              \"useAvroLogicalTypes can only be set with Avro output.\");\n        }\n        checkArgument(\n            !getPropagateSuccessfulStorageApiWrites(),\n            \"withPropagateSuccessfulStorageApiWrites only supported when using storage api writes.\");\n        if (!(getBadRecordRouter() instanceof ThrowingBadRecordRouter)) {\n          LOG.warn(\n              \"Error Handling is partially supported when using FILE_LOADS. Consider using STORAGE_WRITE_API or STORAGE_API_AT_LEAST_ONCE\");\n        }\n\n        // Batch load handles wrapped json string value differently than the other methods. Raise a\n        // warning when applies.\n        ValueProvider<String> jsonSchema = getJsonSchema();\n        if (jsonSchema != null && jsonSchema.isAccessible()) {\n          JsonElement schema = JsonParser.parseString(jsonSchema.get());\n          if (!schema.getAsJsonObject().keySet().isEmpty() && hasJsonTypeInSchema(schema)) {\n            if (rowWriterFactory.getOutputType() == OutputType.JsonTableRow) {\n              LOG.warn(\n                  \"Found JSON type in TableSchema for 'FILE_LOADS' write method. \\n\"\n                      + \"Make sure the TableRow value is a Jackson JsonNode to ensure the read as a \"\n                      + \"JSON type. Otherwise it will read as a raw (escaped) string.\\n\"\n                      + \"See https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-json#limitations \"\n                      + \"for limitations.\");\n            } else if (rowWriterFactory.getOutputType() == OutputType.AvroGenericRecord) {\n              LOG.warn(","sourceCodeStart":4209,"sourceCodeEnd":4245,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java#L4209-L4245","documentation":"Bad-record error handling (withBadRecordRouter / error collection) is only fully implemented for the Storage API write methods. When FILE_LOADS is used with a non-throwing bad record router, only partial error handling is available, so the sink warns users to switch methods for complete error handling.","triggerScenarios":"A FILE_LOADS write whose getBadRecordRouter() is not a ThrowingBadRecordRouter — i.e. a custom/routing BadRecordRouter was configured via withBadRecordRouter(...) while keeping the batch file-load method.","commonSituations":"Teams adding data-quality error collection to an existing batch BigQuery load pipeline instead of migrating to the Storage API write path that fully supports bad-record routing.","solutions":["Switch the write method to Method.STORAGE_WRITE_API or Method.STORAGE_API_AT_LEAST_ONCE for full error handling","Keep the default ThrowingBadRecordRouter if FILE_LOADS must be used and partial handling is unacceptable","Review the partial FILE_LOADS error semantics in the BigQueryIO docs before relying on routed records"],"exampleFix":"// before\nBigQueryIO.writeTableRows().to(table).withMethod(Method.FILE_LOADS).withBadRecordRouter(customRouter)\n// after\nBigQueryIO.writeTableRows().to(table).withMethod(Method.STORAGE_WRITE_API).withBadRecordRouter(customRouter)","handlingStrategy":"validation","validationCode":"if (method == Method.FILE_LOADS && !(badRecordRouter instanceof ThrowingBadRecordRouter)) { /* expect only partial error handling */ }","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Use STORAGE_WRITE_API or STORAGE_API_AT_LEAST_ONCE when bad-record routing matters","Keep the default ThrowingBadRecordRouter for FILE_LOADS pipelines","Read BigQueryIO error-handling docs before adding withBadRecordRouter to batch loads"],"tags":["java","apache-beam","bigquery","error-handling"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}