{"record":{"id":"1183b472cd691d4d","repo":"apache/beam","slug":"batch-type-must-be-pd-series-or-pd-dataframe","errorCode":null,"errorMessage":"batch type must be pd.Series or pd.DataFrame","messagePattern":"batch type must be pd\\.Series or pd\\.DataFrame","errorType":"validation","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/typehints/pandas_type_compatibility.py","lineNumber":149,"sourceCode":"  if fieldtype is not None:\n    return fieldtype\n  elif dtype.kind == 'S':\n    return bytes\n  else:\n    return Any\n\n\n@BatchConverter.register(name=\"pandas\")\ndef create_pandas_batch_converter(\n    element_type: type, batch_type: type) -> BatchConverter:\n  if batch_type == pd.DataFrame:\n    return DataFrameBatchConverter.from_typehints(\n        element_type=element_type, batch_type=batch_type)\n  elif batch_type == pd.Series:\n    return SeriesBatchConverter.from_typehints(\n        element_type=element_type, batch_type=batch_type)\n\n  raise TypeError(\"batch type must be pd.Series or pd.DataFrame\")\n\n\nclass DataFrameBatchConverter(BatchConverter):\n  def __init__(\n      self,\n      element_type: RowTypeConstraint,\n  ):\n    super().__init__(pd.DataFrame, element_type)\n    self._columns = [name for name, _ in element_type._fields]\n\n  @staticmethod\n  def from_typehints(element_type,\n                     batch_type) -> Optional['DataFrameBatchConverter']:\n    assert batch_type == pd.DataFrame\n\n    if not isinstance(element_type, RowTypeConstraint):\n      element_type = RowTypeConstraint.from_user_type(element_type)\n      if element_type is None:","sourceCodeStart":131,"sourceCodeEnd":167,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/typehints/pandas_type_compatibility.py#L131-L167","documentation":"In Beam's pandas batch support, create_pandas_batch_converter builds a BatchConverter that maps between individual elements and pandas batches. Only pd.DataFrame and pd.Series are valid batch types; anything else raises this TypeError. The library deliberately restricts batching to these two pandas containers.","triggerScenarios":"Passing batch_type to create_pandas_batch_converter (directly or through BatchConverter.from_typehints or the @with_batch_types / _unbatch_transform path) as something other than pd.DataFrame or pd.Series, e.g. a string 'DataFrame', a subclass, numpy.ndarray, or None.","commonSituations":"Configuring beam.BatchElements with a custom batch type, or wiring unbatch transforms in a pipeline where the batch type was typo'd or taken from a config instead of the actual pandas class object.","solutions":["Import pandas and pass the class object itself: batch_type=pd.DataFrame or batch_type=pd.Series.","Check for string/config-driven batch types and resolve them to the actual pandas class before calling.","If you need another container (e.g. numpy arrays), use the torch/arrow converters or write a custom BatchConverter subclass instead.","Catch TypeError at converter construction to fail fast with a clearer pipeline error."],"exampleFix":"// before\nconverter = create_pandas_batch_converter(element_type=element_type, batch_type='pd.DataFrame')\n// after\nimport pandas as pd\nconverter = create_pandas_batch_converter(element_type=element_type, batch_type=pd.DataFrame)","handlingStrategy":"validation","validationCode":"import pandas as pd\ndef valid_batch_type(batch_type) -> bool:\n    return batch_type in (pd.DataFrame, pd.Series)\n","typeGuard":"def is_pandas_batch_type(batch_type) -> bool:\n    import pandas as pd\n    return batch_type in (pd.DataFrame, pd.Series)\n","tryCatchPattern":"try:\n    converter = create_pandas_batch_converter(element_type=et, batch_type=bt)\nexcept TypeError as e:\n    raise ValueError(f'Unsupported batch type {bt!r}; use pd.DataFrame or pd.Series') from e\n","preventionTips":["Always pass pandas class objects, never strings or names from config.","Check the batch type table before wiring BatchElements into a pipeline.","Use pandas classes imported at module scope to avoid None placeholders.","Write a smoke test constructing the converter at pipeline-build time."],"tags":["python","apache-beam","pandas","batching"],"backgroundTag":"invalid-argument-value","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}