{"record":{"id":"c413c3daa1f82d1e","repo":"apache/beam","slug":"schema-update-options-is-only-supported-when-writing-to","errorCode":null,"errorMessage":"schema_update_options is only supported when writing to BigQuery with FILE_LOADS.","messagePattern":"schema_update_options is only supported when writing to BigQuery with FILE_LOADS\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/gcp/bigquery.py","lineNumber":2354,"sourceCode":"\n    if (isinstance(self.table_reference, TableReference) and\n        self.table_reference.projectId is None):\n      self.table_reference.projectId = pcoll.pipeline.options.view_as(\n          GoogleCloudOptions).project\n\n    # TODO(pabloem): Use a different method to determine if streaming or batch.\n    is_streaming_pipeline = p.options.view_as(StandardOptions).streaming\n\n    if not is_streaming_pipeline and self.with_auto_sharding:\n      raise ValueError(\n          'with_auto_sharding is not applicable to batch pipelines.')\n\n    experiments = p.options.view_as(DebugOptions).experiments or []\n    method_to_use = self._compute_method(experiments, is_streaming_pipeline)\n\n    if (self.schema_update_options is not None and\n        method_to_use != WriteToBigQuery.Method.FILE_LOADS):\n      raise ValueError(\n          'schema_update_options is only supported when writing to BigQuery '\n          'with FILE_LOADS.')\n\n    if method_to_use == WriteToBigQuery.Method.STREAMING_INSERTS:\n      if self.schema == SCHEMA_AUTODETECT:\n        raise ValueError(\n            'Schema auto-detection is not supported for streaming '\n            'inserts into BigQuery. Only for File Loads.')\n\n      if self.triggering_frequency is not None and not self.with_auto_sharding:\n        raise ValueError(\n            'triggering_frequency with STREAMING_INSERTS can only be used with '\n            'with_auto_sharding=True.')\n\n      if self._max_insert_payload_size > MAX_INSERT_PAYLOAD_SIZE:\n        raise ValueError(\n            'max_insert_payload_size can only go up to '\n            f'{MAX_INSERT_PAYLOAD_SIZE} bytes, as per BigQuery quota limits: '","sourceCodeStart":2336,"sourceCodeEnd":2372,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/gcp/bigquery.py#L2336-L2372","documentation":"schema_update_options (e.g. ALLOW_FIELD_ADDITION / ALLOW_FIELD_RELAXATION) are implemented by BigQuery load jobs only. When the chosen write method is not FILE_LOADS (e.g. STREAMING_INSERTS) the schema update options cannot be honored, so expand() raises ValueError rather than silently ignoring them.","triggerScenarios":"WriteToBigQuery with schema_update_options=['ALLOW_FIELD_ADDITION'] while method resolves to STREAMING_INSERTS (explicitly, by default in batch, or via _compute_method).","commonSituations":"Defaulting to STREAMING_INSERTS but copying schema_update_options from an existing file-loads pipeline; forgetting that STREAMING_INSERTS is the batch default in older Beam versions.","solutions":["Pass method=WriteToBigQuery.Method.FILE_LOADS together with schema_update_options.","Remove schema_update_options if you must keep STREAMING_INSERTS.","Enable the storage write API (experiments/use_storage_write_api) only if your Beam version supports schema update there."],"exampleFix":"// before\nbeam.io.WriteToBigQuery(table, schema_update_options=['ALLOW_FIELD_ADDITION'])\n// after\nbeam.io.WriteToBigQuery(table, method=beam.io.WriteToBigQuery.Method.FILE_LOADS, schema_update_options=['ALLOW_FIELD_ADDITION'])","handlingStrategy":"validation","validationCode":"if schema_update_options and method != WriteToBigQuery.Method.FILE_LOADS:\n    raise ValueError('schema_update_options require FILE_LOADS')","typeGuard":null,"tryCatchPattern":"try:\n    _ = beam.io.WriteToBigQuery(table, method=method, schema_update_options=opts)\nexcept ValueError as e:\n    logger.error('%s; switch to FILE_LOADS or drop schema_update_options', e)","preventionTips":["Pair schema_update_options explicitly with method=FILE_LOADS","Remember STREAMING_INSERTS may be the default in batch","Validate the method/options combination in a shared helper"],"tags":["bigquery","python","apache-beam","file-loads"],"backgroundTag":"conflicting-config-options","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}