apache/beam · error · ImportError

GCS not available; please install apache_beam[gcp]

Error message

GCS not available; please install apache_beam[gcp]

What it means

render_one reads the pipeline proto from --pipeline_proto; if the path is a gs:// URL, it needs the GCS IO module from the apache_beam[gcp] extra. When that optional dependency is absent, it raises ImportError telling the user to install apache_beam[gcp]. Beam keeps GCP support as an optional extra to limit the default install size.

Source

Thrown at sdks/python/apache_beam/runners/render.py:528

  run_server(options)


def render_one(options):
  if options.pipeline_proto == '-':
    content = sys.stdin.buffer.read()
    if content[0] == b'{':
      ext = '.json'
    else:
      try:
        content.decode('utf-8')
        ext = '.textproto'
      except UnicodeDecodeError:
        ext = '.pb'
  else:
    if options.pipeline_proto.startswith('gs://'):
      if gcsio is None:
        raise ImportError('GCS not available; please install apache_beam[gcp]')
      open_fn = gcsio.GcsIO().open
    else:
      open_fn = open

    with open_fn(options.pipeline_proto, 'rb') as fin:
      content = fin.read()
    ext = os.path.splitext(options.pipeline_proto)[-1]

  if ext == '.textproto':
    pipeline_proto = text_format.Parse(content, beam_runner_api_pb2.Pipeline())
  elif ext == '.json':
    pipeline_proto = json_format.Parse(content, beam_runner_api_pb2.Pipeline())
  else:
    pipeline_proto = beam_runner_api_pb2.Pipeline()
    pipeline_proto.ParseFromString(content)

  RenderRunner().run_portable_pipeline(
      pipeline_proto, pipeline_options.PipelineOptions(**vars(options)))

View on GitHub (pinned to 12126d8942)

Solutions

1. pip install apache_beam[gcp] to bring in the GCS dependencies
2. Download the proto locally (gsutil cp) and point --pipeline_proto at the local file
3. Verify the installed beam package includes google-cloud-storage (pip show apache-beam)

Example fix

# before
pip install apache-beam
--pipeline_proto=gs://my-bucket/pipeline.pb  # ImportError
# after
pip install 'apache-beam[gcp]'
--pipeline_proto=gs://my-bucket/pipeline.pb

from apache_beam.options.pipeline_options import PipelineOptions
if pipeline_proto.startswith('gs://'):
    import importlib.util
    if importlib.util.find_spec('google.cloud.storage') is None:
        raise SystemExit("Run: pip install 'apache_beam[gcp]' to read gs:// protos")

def can_read_proto_source(path: str) -> bool:
    import importlib.util
    return (not path.startswith('gs://')) or importlib.util.find_spec('apache_beam.io.gcp.internal.gcsio') is not None

try:
    render(proto_path)
except ImportError as e:
    if 'GCS not available' in str(e):
        subprocess.run(['gsutil', 'cp', proto_path, 'local.pb'], check=True)
        render('local.pb')  # fallback: local copy
    else:
        raise

When it happens

Trigger: Running the render entry point with --pipeline_proto=gs://bucket/path on an environment where the gcsio module is None (google-cloud-storage not installed).

Common situations: Slim virtualenv with only base apache_beam; trying to render a Dataflow-exported proto straight from GCS; container images stripped of GCP extras.

Understand the failure class

Background: "X is not installed. Please install it with pip install Y": missing optional dependency errors — ImportError/ValueError raised when a library's optional extra was never installed — this error's family across 22 libraries.

Related errors

• Found no files that match {self.path!r}
• src and dst files do not exist. src: %s, dst: %s
• Doubly compressed files not supported.
• langchain is required to use LangChainChunkerPlease install
• sentence-transformers is required to use HuggingfaceTextEmbe

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7c3d659e3c98bbd9. Report an issue: GitHub.