apache/beam · error · ValueError

Only dataframes with single rows are supported.

Error message

Only dataframes with single rows are supported.

What it means

Raised in the KeyedModelHandler's run_inference for sklearn when a pandas DataFrame in the batch has more (or fewer) than one row. The sklearn dataframe inference path only supports single-row dataframes per element, and each batched dataframe is validated with dataframe.shape[0] != 1.

Source

Thrown at sdks/python/apache_beam/ml/inference/sklearn_inference.py:322

      inference_args: Optional[dict[str, Any]] = None
  ) -> Iterable[PredictionResult]:
    """
    Runs inferences on a batch of pandas dataframes.

    Args:
      batch: A sequence of examples as numpy arrays. They should
        be single examples.
      model: A dataframe model or pipeline. Must implement predict(X).
        Where the parameter X is a pandas dataframe.
      inference_args: Any additional arguments for an inference.

    Returns:
      An Iterable of type PredictionResult.
    """
    # sklearn_inference currently only supports single rowed dataframes.
    for dataframe in iter(batch):
      if dataframe.shape[0] != 1:
        raise ValueError('Only dataframes with single rows are supported.')

    predictions, splits = self._model_inference_fn(model, batch, inference_args)

    return utils._convert_to_result(
        splits, predictions, model_id=self._model_uri)

  def get_num_bytes(self, batch: Sequence[pandas.DataFrame]) -> int:
    """
    Returns:
      The number of bytes of data for a batch.
    """
    return sum(df.memory_usage(deep=True).sum() for df in batch)

  def get_metrics_namespace(self) -> str:
    """
    Returns:
       A namespace for metrics collected by the RunInference transform.
    """

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure each input element is a DataFrame with exactly one row
2. Set batch_size=1 (or use the default) with the dataframe-based sklearn handler
3. Switch to the numpy-based handler if your inputs are arrays and you want multi-row batching

Example fix

// before
handler = SklearnModelHandlerKeyedModel(model_uri=uri)
_ = pcoll | RunInference(handler)  # elements are multi-row DataFrames
// after
df_single = df.iloc[[0]]  # one row per element
handler = SklearnModelHandlerKeyedModel(model_uri=uri, batch_size=1)

def validate_dataframe_inputs(elements):
    for k, df in elements:
        if getattr(df, 'shape', (0,))[0] != 1:
            raise ValueError(f'Element for key {k!r} has {df.shape[0]} rows; sklearn handler supports exactly 1 row per DataFrame.')

def is_single_row_dataframe(x) -> bool:
    import pandas as pd
    return isinstance(x, pd.DataFrame) and x.shape[0] == 1

try:
    result = pcoll | RunInference(handler)
except ValueError as e:
    if 'single rows' in str(e):
        raise RuntimeError('Reshape inputs to one row per DataFrame or lower batch_size') from e
    raise

When it happens

Trigger: Using SklearnModelHandlerKeyedModel (dataframe input) with batch_size > 1 or elements that are multi-row DataFrames; batching logic combining multiple rows into one DataFrame before run_inference is invoked.

Common situations: Setting batch_size larger than 1 with dataframe inputs; preprocessing code that concatenates rows into a single DataFrame per key; misusing the keyed dataframe handler with tensors/arrays meant for the numpy handler.

Related errors

• batch type must be pd.Series or pd.DataFrame
• Element type must be compatible with Beam Schemas (https://b
• concat(ignore_index)
• concat(levels)
• Could not import joblib in this execution environment. For h

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/2b8412efd23a27a4. Report an issue: GitHub.