{"record":{"id":"6dbe7e9b4de0d823","repo":"apache/beam","slug":"google-cloud-dataflow-runner-not-available-please-install-6dbe7e","errorCode":null,"errorMessage":"Google Cloud Dataflow runner not available, please install apache_beam[gcp]","messagePattern":"Google Cloud Dataflow runner not available, please install apache_beam\\[gcp\\]","errorType":"exception","errorClass":"ImportError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/runners/runner.py","lineNumber":87,"sourceCode":"\n  Raises:\n    RuntimeError: if an invalid runner name is used.\n  \"\"\"\n\n  # Get the qualified runner name by using the lower case runner name. If that\n  # fails try appending the name with 'runner' and check if it matches.\n  # If that also fails, use the given runner name as is.\n  runner_name = _RUNNER_MAP.get(\n      runner_name.lower(),\n      _RUNNER_MAP.get(runner_name.lower() + 'runner', runner_name))\n\n  if '.' in runner_name:\n    module, runner = runner_name.rsplit('.', 1)\n    try:\n      return getattr(importlib.import_module(module), runner)()\n    except ImportError:\n      if 'dataflow' in runner_name.lower():\n        raise ImportError(\n            'Google Cloud Dataflow runner not available, '\n            'please install apache_beam[gcp]')\n      elif 'interactive' in runner_name.lower():\n        raise ImportError(\n            'Interactive runner not available, '\n            'please install apache_beam[interactive]')\n      else:\n        raise\n  else:\n    raise ValueError(\n        'Unexpected pipeline runner: %s. Valid values are %s '\n        'or the fully qualified name of a PipelineRunner subclass.' %\n        (runner_name, ', '.join(StandardOptions.KNOWN_RUNNER_NAMES)))\n\n\nclass PipelineRunner(object):\n  \"\"\"A runner of a pipeline object.\n","sourceCodeStart":69,"sourceCodeEnd":105,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/runner.py#L69-L105","documentation":"PipelineOptions.create_runner resolves a runner by dotted module path; when the import fails with ImportError and the name mentions 'dataflow', it re-raises a clearer ImportError explaining that the Dataflow runner requires the apache_beam[gcp] extra. This translates a raw module-import failure into an actionable install hint.","triggerScenarios":"Passing --runner=DataflowRunner (or --runner=apache_beam.runners.dataflow.DataflowRunner) without google-cloud dependencies installed, so importing the dataflow runner module raises ImportError.","commonSituations":"Local venvs with only apache-beam base install; CI jobs that submit to Dataflow without apache_beam[gcp]; typo'd runner class names alongside genuinely missing GCP extras.","solutions":["pip install apache_beam[gcp] before running with DataflowRunner","Verify the runner class name is spelled correctly (DataflowRunner)","Check that importlib can find the module: python -c 'from apache_beam.runners.dataflow import DataflowRunner'"],"exampleFix":"# before\npip install apache-beam\n--runner=DataflowRunner  # ImportError: not available\n# after\npip install 'apache-beam[gcp]'\n--runner=DataflowRunner","handlingStrategy":"validation","validationCode":"import importlib.util\nif runner_name.endswith('DataflowRunner') and importlib.util.find_spec('google.cloud.storage') is None:\n    raise SystemExit(\"Install GCP extras first: pip install 'apache_beam[gcp]'\")","typeGuard":"def runner_importable(runner_name: str) -> bool:\n    import importlib\n    if '.' not in runner_name:\n        return True\n    module, runner = runner_name.rsplit('.', 1)\n    try:\n        importlib.import_module(module)\n        return True\n    except ImportError:\n        return False","tryCatchPattern":"try:\n    runner = PipelineOptions(argv).runner  # or create_runner(runner_name)\nexcept ImportError as e:\n    if 'apache_beam[gcp]' in str(e):\n        subprocess.run(['pip', 'install', 'apache_beam[gcp]'], check=True)\n        runner = create_runner(runner_name)\n    else:\n        raise","preventionTips":["Install apache_beam[gcp] in any environment that runs Dataflow jobs","Fully qualify runner names to catch typos separately from missing deps","Add an import check for the runner module early in launch scripts"],"tags":["python","apache-beam","dataflow","runner","optional-dependency"],"backgroundTag":"missing-optional-dependency","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}