apache/beam · error · ValueError

nrows not yet supported

Error message

nrows not yet supported

What it means

In the Beam DataFrame API's read_csv, the nrows parameter is not implemented: passing it raises this guard error at call time. The deferred CSV reader supports streaming/splittable reads but not the row-limit option; read the data and use head()/limit downstream instead.

Source

Thrown at sdks/python/apache_beam/dataframe/io.py:95

      to false or leave it unspecified.
      """
  if table is None:
    raise ValueError("Please specify a BigQuery table to read from.")
  elif len(kwargs) > 0:
    raise ValueError(
        f"Encountered unsupported parameter(s) in read_gbq: {kwargs.keys()!r}"
        "")
  return _ReadGbq(table, dataset, project_id, use_bqstorage_api)


@frame_base.with_docs_from(pd)
def read_csv(path, *args, splittable=False, binary=True, **kwargs):
  """If your files are large and records do not contain quoted newlines, you may
  pass the extra argument ``splittable=True`` to enable dynamic splitting for
  this read on newlines. Using this option for records that do contain quoted
  newlines may result in partial records and data corruption."""
  if 'nrows' in kwargs:
    raise ValueError('nrows not yet supported')
  filename_column = kwargs.pop('filename_column', None)
  return _ReadFromPandas(
      pd.read_csv,
      path,
      args,
      kwargs,
      incremental=True,
      binary=binary,
      splitter=_TextFileSplitter(args, kwargs) if splittable else None,
      filename_column=filename_column)


def _as_pc(df, label=None):
  from apache_beam.dataframe import convert  # avoid circular import

  # TODO(roberwb): Amortize the computation for multiple writes?
  return convert.to_pcollection(df, yield_elements='pandas', label=label)

View on GitHub (pinned to 12126d8942)

Solutions

1. Remove nrows and read the whole file (or a narrower file path).
2. Sample after the read: read then use .head(n) or .limit via Beam transforms.
3. If you only need a preview, read the file with plain pandas instead of Beam.

Example fix

// before
df = read_csv('data.csv', nrows=1000)
// after
df = read_csv('data.csv')
df = df.head(1000)  # or apply a limit downstream

if 'nrows' in kwargs:
    del kwargs['nrows']  # or sample downstream instead

try:
    df = read_csv(path, **kwargs)
except ValueError as e:
    if 'nrows' in str(e):
        kwargs.pop('nrows', None)
        df = read_csv(path, **kwargs).head(nrows)
    else:
        raise

When it happens

Trigger: Calling beam.dataframe.io.read_csv('f.csv', nrows=100) or forwarding a kwargs dict that contains nrows from a shared pandas configuration.

Common situations: Reusing pandas read_csv exploration code for sampling/head-of-file reads in a Beam pipeline; a shared kwargs dict built for pandas passed unchanged to Beam.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Encountered an Atomic type that is not currently supported b
• Cannot specify both 'labels' and 'index'/'columns'
• axis must be one of (0, 1, 'index', 'columns'), got '%s'
• groupby(as_index=False)
• You have to supply one of 'by' and 'level'

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/c614f4327b9f9ffe. Report an issue: GitHub.