{"record":{"id":"871072a2bdc63faa","repo":"apache/beam","slug":"could-not-generate-protos-due-to-error-s","errorCode":null,"errorMessage":"Could not generate protos due to error: %s","messagePattern":"Could not generate protos due to error: (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"sdks/python/setup.py","lineNumber":259,"sourceCode":"      # make sure we already generated protos\n      pb2_files = list(\n          find_by_ext(\n              os.path.join(cwd, 'apache_beam', 'portability', 'api'),\n              '_pb2.py'))\n      if not pb2_files:\n        raise RuntimeError(\n            'protobuf files are not generated. '\n            'Please generate pb2 files')\n\n      warnings.warn('Skipping proto generation as they are already generated.')\n      return\n    out = subprocess.run(\n        [sys.executable, os.path.join(cwd, 'gen_protos.py'), '--no-force'],\n        capture_output=True,\n        check=True)\n    print(out.stdout)\n  except subprocess.CalledProcessError as err:\n    raise RuntimeError('Could not generate protos due to error: %s', err.stderr)\n\n\ndef copy_tests_from_docs():\n  python_root = os.path.abspath(os.path.dirname(__file__))\n  docs_src = os.path.normpath(\n      os.path.join(\n          python_root, '../../website/www/site/content/en/documentation/sdks'))\n  docs_dest = os.path.normpath(\n      os.path.join(python_root, 'apache_beam/yaml/docs'))\n  if os.path.exists(docs_src):\n    shutil.rmtree(docs_dest, ignore_errors=True)\n    os.mkdir(docs_dest)\n    for path in glob.glob(os.path.join(docs_src, 'yaml*.md')):\n      shutil.copy(path, docs_dest)\n  else:\n    warnings.warn(\n        f'Could not locate yaml docs source directory {docs_src}. '\n        f'Skipping copying tests from docs.')","sourceCodeStart":241,"sourceCodeEnd":277,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/setup.py#L241-L277","documentation":"When protos must be generated, setup.py shells out to gen_protos.py with check=True; if that subprocess exits nonzero (CalledProcessError), setup.py re-raises as RuntimeError including the subprocess stderr. The real failure cause (protoc missing, syntax error in .proto, network fetch failure) is in err.stderr.","triggerScenarios":"Running a build/install that triggers generate_protos_first when gen_protos.py fails: protoc/grpcio-tools not installed or incompatible version, malformed proto definitions, or grpc plugin errors.","commonSituations":"Build environments lacking protoc; mismatched grpcio-tools versions after a Beam upgrade; proto generation flaky in restricted CI sandboxes without network access.","solutions":["Read err.stderr in the message for the root cause and fix it (usually a protoc or grpcio-tools issue)","Run `python gen_protos.py` manually to see full output and iterate","Pin/install compatible tools: pip install grpcio-tools mypy-protobuf types-protobuf","Ensure protoc is on PATH and matches the protobuf runtime version"],"exampleFix":"# before\n$ pip install -e .\nRuntimeError: Could not generate protos due to error: b'protoc not found...'\n# after\n$ pip install grpcio-tools\n$ python gen_protos.py --no-force\n$ pip install -e .","handlingStrategy":"try-catch","validationCode":"import shutil\nif shutil.which('protoc') is None:\n    raise RuntimeError('protoc is required to generate Beam protos')","typeGuard":null,"tryCatchPattern":"try:\n    generate_protos_first()\nexcept RuntimeError as e:\n    if 'Could not generate protos' in str(e):\n        install_proto_toolchain()  # pip install grpcio-tools, protoc\n        generate_protos_first()\n    else:\n        raise","preventionTips":["Pre-install protoc and matching grpcio-tools in build images","Run gen_protos.py manually once after setup to surface errors early","Pin grpcio-tools/protobuf versions to the Beam SDK requirements"],"tags":["python","protobuf","subprocess","build"],"backgroundTag":"command-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}