apache/beam · error · ImportError

Please install apache_beam[dataframe]

Error message

Please install apache_beam[dataframe]

What it means

When pandas (and hence the csv/json DataFrame I/O) is not installed, apache_beam.io.textio stubs ReadFromCsv/WriteToCsv/ReadFromJson/WriteToJson with the no_pandas function, which raises ImportError on any use. It exists so the module imports cleanly without the optional dependency.

Source

Thrown at sdks/python/apache_beam/io/textio.py:1113

        as opposed to the entire file being a valid JSON object or list.
        Defaults to True if orient is 'records' (unlike Pandas).
      **kwargs: Extra arguments passed to `pandas.Dataframe.to_json`
        (see below).
    """
    from apache_beam.dataframe.io import WriteViaPandas
    if num_shards is not None:
      kwargs['num_shards'] = num_shards
    if file_naming is not None:
      kwargs['file_naming'] = file_naming
    if lines is None:
      lines = orient == 'records'
    return 'WriteToJson' >> WriteViaPandas(
        'json', path, orient=orient, lines=lines, **kwargs)

except ImportError:

  def no_pandas(*args, **kwargs):
    raise ImportError('Please install apache_beam[dataframe]')

  for transform in ('ReadFromCsv', 'WriteToCsv', 'ReadFromJson', 'WriteToJson'):
    globals()[transform] = no_pandas

View on GitHub (pinned to 12126d8942)

Solutions

1. Install pandas: pip install apache_beam[dataframe] (or pip install pandas)
2. Pin pandas in requirements.txt for Dataflow workers
3. Use ReadFromText with a custom coder/parse function if pandas is not desired

Example fix

// before
beam.io.ReadFromCsv('gs://bucket/data.csv')
// after
# requirements.txt: apache-beam[dataframe]
beam.io.ReadFromCsv('gs://bucket/data.csv')

try:
    import pandas  # noqa
    PANDAS_AVAILABLE = True
except ImportError:
    PANDAS_AVAILABLE = False

try:
    rows = beam.io.ReadFromCsv(path)
except ImportError:
    # fallback: plain text + manual parsing
    rows = beam.io.ReadFromText(path) | beam.Map(parse_csv_line)

When it happens

Trigger: Calling ReadFromCsv, WriteToCsv, ReadFromJson or WriteToJson in an environment where pandas is absent (the apache_beam.io.textio import succeeded but the pandas-dependent block hit ImportError).

Common situations: Minimal docker images or runners with apache-beam installed without the [dataframe] or [gcp] extras; CI environments without pandas.

Understand the failure class

Background: "X is not installed. Please install it with pip install Y": missing optional dependency errors — ImportError/ValueError raised when a library's optional extra was never installed — this error's family across 22 libraries.

Related errors

• Delimiter must be a non-empty bytes sequence.
• Delimiter must not self-overlap.
• escapechar must be bytes of size 1: '%s'
• Could not find python-snappy, google-crc32c, or crcmod. To a
• Performing a numeric aggregation, {base_func!r}, on Series {

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/02c194432a43bedf. Report an issue: GitHub.