{"record":{"id":"e4f67e4aa7b81385","repo":"apache/beam","slug":"currently-dynamic-clustering-and-timepartitioning-is-not","errorCode":null,"errorMessage":"Currently, dynamic clustering and timepartitioning is not supported for STORAGE_WRITE_API write method.","messagePattern":"Currently, dynamic clustering and timepartitioning is not supported for STORAGE_WRITE_API write method\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/gcp/bigquery.py","lineNumber":2826,"sourceCode":"                        StorageWriteToBigQuery.RECORD: row[1]\n                    })).with_output_types(\n                        RowTypeConstraint.from_fields([\n                            (StorageWriteToBigQuery.DESTINATION, str),\n                            (StorageWriteToBigQuery.RECORD, input.element_type)\n                        ])))\n      # otherwise, convert to Beam Rows\n      else:\n        input_beam_rows = (\n            input_rows\n            | \"Convert dict to Beam Row\" >> self.ConvertToBeamRows(\n                schema, True, self._type_overrides).with_output_types())\n      # communicate to Java that this write should use dynamic destinations\n      table = StorageWriteToBigQuery.DYNAMIC_DESTINATIONS\n\n    clustering_fields = []\n    if self.additional_bq_parameters:\n      if callable(self.additional_bq_parameters):\n        raise NotImplementedError(\n            \"Currently, dynamic clustering and timepartitioning is not \"\n            \"supported for STORAGE_WRITE_API write method.\")\n      clustering_fields = (\n          self.additional_bq_parameters.get(\"clustering\", {}).get(\"fields\", []))\n\n    output = (\n        input_beam_rows\n        | SchemaAwareExternalTransform(\n            identifier=StorageWriteToBigQuery.IDENTIFIER,\n            expansion_service=self._expansion_service,\n            rearrange_based_on_discovery=True,\n            table=table,\n            create_disposition=self._create_disposition,\n            write_disposition=self._write_disposition,\n            triggering_frequency_seconds=self._triggering_frequency,\n            auto_sharding=self._with_auto_sharding,\n            num_streams=self._num_storage_api_streams,\n            use_at_least_once_semantics=self._use_at_least_once,","sourceCodeStart":2808,"sourceCodeEnd":2844,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/gcp/bigquery.py#L2808-L2844","documentation":"StorageWriteToBigQuery.expand() raises NotImplementedError when additional_bq_parameters is callable. A callable implies per-destination dynamic time partitioning/clustering, which the Storage Write API path does not support; it only reads static 'clustering' fields from a dict.","triggerScenarios":"WriteToBigQuery(method=STORAGE_WRITE_API, additional_bq_parameters=lambda destination: {...}) expanded.","commonSituations":"Reusing the callable additional_bq_parameters pattern from FILE_LOADS dynamic destinations when migrating to STORAGE_WRITE_API.","solutions":["Pass additional_bq_parameters as a static dict, e.g. {'timePartitioning': {...}, 'clustering': {'fields': [...]}}.","Split the pipeline into multiple WriteToBigQuery transforms, one per partitioning configuration.","Use FILE_LOADS if dynamic per-destination clustering/partitioning is required."],"exampleFix":"// before\nWriteToBigQuery(method='STORAGE_WRITE_API', additional_bq_parameters=lambda dest: {'clustering': {'fields': ['date']}})\n// after\nWriteToBigQuery(method='STORAGE_WRITE_API', additional_bq_parameters={'clustering': {'fields': ['date']}})","handlingStrategy":"validation","validationCode":"if method == 'STORAGE_WRITE_API' and callable(additional_bq_parameters):\n    raise ValueError('STORAGE_WRITE_API needs static additional_bq_parameters')","typeGuard":"def is_static_bq_params(p):\n    return isinstance(p, dict) or p is None","tryCatchPattern":null,"preventionTips":["Use a static dict for partitioning/clustering with STORAGE_WRITE_API.","Split per-destination configs into separate writes instead of callables."],"tags":["apache-beam","bigquery","storage-write-api","not-implemented","dynamic-destinations"],"backgroundTag":"method-not-implemented","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}