apache/beam · error · ValueError

Write disposition is not supported for streaming inserts to…

Error message

Write disposition %s is not supported for streaming inserts to BigQuery

What it means

BigQueryWriteFn (STREAMING_INSERTS method) rejects write dispositions WRITE_EMPTY and WRITE_TRUNCATE. Those dispositions are table-level load/write semantics evaluated at job creation, which do not apply to per-row streaming inserts; only WRITE_APPEND makes sense for the streaming path, so the constructor raises ValueError.

Solutions

Use write_disposition=BigQueryDisposition.WRITE_APPEND with STREAMING_INSERTS.
Switch the method to FILE_LOADS if you actually need WRITE_TRUNCATE/WRITE_EMPTY semantics.
Parameterize the disposition per method and validate before constructing the transform.

Example fix

// before
beam.io.WriteToBigQuery(table, method='STREAMING_INSERTS', write_disposition='WRITE_TRUNCATE')
// after
beam.io.WriteToBigQuery(table, method='STREAMING_INSERTS', write_disposition='WRITE_APPEND')

Defensive patterns

Strategy: validation

Validate before calling

if method == WriteToBigQuery.Method.STREAMING_INSERTS and write_disposition in ('WRITE_EMPTY', 'WRITE_TRUNCATE'):
    raise ValueError('streaming inserts require WRITE_APPEND')

Type guard

def streaming_disposition_ok(d):
    return d in (None, BigQueryDisposition.WRITE_APPEND)

Try / catch

try:
    _ = beam.io.WriteToBigQuery(table, method='STREAMING_INSERTS', write_disposition=disp)
except ValueError as e:
    logger.error('Invalid streaming write disposition: %s', e)

Prevention

Use WRITE_APPEND for streaming inserts
Keep separate configs for streaming and file-load pipelines
Centralize BigQuery writer construction with method-aware validation

When it happens

Trigger: Constructing WriteToBigQuery(method=STREAMING_INSERTS, write_disposition=WRITE_TRUNCATE or WRITE_EMPTY) — including the default STREAMING_INSERTS path with an explicitly set non-append disposition.

Common situations: Reusing a config shared between file-load and streaming pipelines; switching a pipeline from FILE_LOADS to STREAMING_INSERTS while keeping write_disposition='WRITE_TRUNCATE'.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/c4868c00cca4bc04. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/gcp/bigquery.py:1546

        destination. If not, perform best-effort batching per destination within
        a bundle.
      ignore_unknown_columns: Accept rows that contain values that do not match
        the schema. The unknown values are ignored. Default is False,
        which treats unknown values as errors. See reference:
        https://cloud.google.com/bigquery/docs/reference/rest/v2/tabledata/insertAll
      max_retries: The number of times that we will retry inserting a group of
        rows into BigQuery. By default, we retry 10000 times with exponential
        backoffs (effectively retry forever).
      max_insert_payload_size: The maximum byte size for a BigQuery legacy
        streaming insert payload.
    """
    self.schema = schema
    self.test_client = test_client
    self.create_disposition = create_disposition
    self.write_disposition = write_disposition
    if write_disposition in (BigQueryDisposition.WRITE_EMPTY,
                             BigQueryDisposition.WRITE_TRUNCATE):
      raise ValueError(
          'Write disposition %s is not supported for'
          ' streaming inserts to BigQuery' % write_disposition)
    self._rows_buffer = []
    self._reset_rows_buffer()

    self._total_buffered_rows = 0
    self.kms_key = kms_key
    self._max_batch_size = batch_size or BigQueryWriteFn.DEFAULT_MAX_BATCH_SIZE
    self._max_buffered_rows = (
        max_buffered_rows or BigQueryWriteFn.DEFAULT_MAX_BUFFERED_ROWS)
    self._retry_strategy = retry_strategy or RetryStrategy.RETRY_ALWAYS
    self.ignore_insert_ids = ignore_insert_ids
    self.with_batched_input = with_batched_input

    self.additional_bq_parameters = additional_bq_parameters or {}

    # accumulate the total time spent in exponential backoff
    self._throttled_secs = Metrics.counter(

View on GitHub (pinned to 12126d8942)