{"record":{"id":"1caf8914a3bbc482","repo":"apache/beam","slug":"writing-with-dynamic-schemas-is-not-supported-for-this-write","errorCode":null,"errorMessage":"Writing with dynamic schemas is not supported for this write method.","messagePattern":"Writing with dynamic schemas is not supported for this write method\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/gcp/bigquery.py","lineNumber":2767,"sourceCode":"    self._num_storage_api_streams = num_storage_api_streams\n    self._use_cdc_writes = use_cdc_writes\n    self._primary_key = primary_key\n    self._big_lake_configuration = big_lake_configuration\n    self._type_overrides = type_overrides\n    self._expansion_service = expansion_service or BeamJarExpansionService(\n        'sdks:java:io:google-cloud-platform:expansion-service:build')\n\n  def expand(self, input):\n    if self._schema is None:\n      try:\n        schema = schema_from_element_type(input.element_type)\n        is_rows = True\n      except TypeError as exn:\n        raise ValueError(\n            \"A schema is required in order to prepare rows \"\n            \"for writing with STORAGE_WRITE_API.\") from exn\n    elif callable(self._schema):\n      raise NotImplementedError(\n          \"Writing with dynamic schemas is not \"\n          \"supported for this write method.\")\n    elif isinstance(self._schema, vp.ValueProvider):\n      schema = self._schema.get()\n      is_rows = False\n    else:\n      schema = self._schema\n      is_rows = False\n\n    table = bigquery_tools.get_hashable_destination(self._table)\n\n    # if writing to one destination, just convert to Beam rows and send over\n    if not callable(table):\n      if is_rows:\n        input_beam_rows = input\n      else:\n        input_beam_rows = (\n            input","sourceCodeStart":2749,"sourceCodeEnd":2785,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/gcp/bigquery.py#L2749-L2785","documentation":"StorageWriteToBigQuery.expand() raises NotImplementedError when schema is a callable, because the Storage Write API path resolves the schema once and cannot compute a schema dynamically per element or destination.","triggerScenarios":"WriteToBigQuery(method=STORAGE_WRITE_API, schema=lambda destination: ...) expanded.","commonSituations":"Migrating from FILE_LOADS (which supports callable schemas for dynamic destinations) to STORAGE_WRITE_API without flattening the callable.","solutions":["Provide a static schema (string, dict, or ValueProvider) instead of a callable.","Resolve the schema yourself before the write (e.g. with a branch per destination and separate WriteToBigQuery transforms).","Use FILE_LOADS or STREAMING_INSERTS which support dynamic schemas."],"exampleFix":"// before\nWriteToBigQuery(method='STORAGE_WRITE_API', schema=lambda dest: schema_for(dest))\n// after\nWriteToBigQuery(method='STORAGE_WRITE_API', schema=schema_for(DEFAULT_DEST))","handlingStrategy":"validation","validationCode":"if method == 'STORAGE_WRITE_API' and callable(schema):\n    raise ValueError('STORAGE_WRITE_API needs a static schema')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Resolve schemas per destination before the write using branch transforms.","Document that dynamic schemas are FILE_LOADS/STREAMING_INSERTS only."],"tags":["apache-beam","bigquery","storage-write-api","not-implemented"],"backgroundTag":"method-not-implemented","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}