apache/beam · error · RuntimeError

Protoc returned non-zero status (see logs for details)

Error message

Protoc returned non-zero status (see logs for details): %s

What it means

After invoking protoc.main() with Beam's proto files (including grpc_python_out), gen_protos.py checks the return code. A non-zero status means protoc failed to compile the protos; the actual errors are in the logs, so this generic RuntimeError just reports the code.

Solutions

Rerun with logging enabled and read the protoc error lines printed before this exception.
Pin/use a protoc version matching the protobuf Python package expected by Beam.
Restore pristine .proto files from git (git checkout -- model/ sdks/python/proto) and retry.

Example fix

// before
python gen_protos.py 2>/dev/null
// after
python gen_protos.py 2>&1 | tee protoc.log  # inspect protoc errors above the RuntimeError

Defensive patterns

Strategy: try-catch

Validate before calling

import subprocess
v = subprocess.run(['protoc', '--version'], capture_output=True, text=True)
assert v.returncode == 0 and 'libprotoc' in v.stdout, 'protoc missing or incompatible'

Type guard

def protoc_works():
    import subprocess
    r = subprocess.run(['protoc', '--version'], capture_output=True)
    return r.returncode == 0

Try / catch

try:
    generate_proto_files()
except RuntimeError as e:
    if 'Protoc returned non-zero status' in str(e):
        # inspect protoc logs emitted earlier, fix proto sources or toolchain version, then retry

Prevention

Run generation with verbose logging and read protoc output before the exception.
Keep protoc and the protobuf Python package versions aligned.
Avoid hand-editing generated or source .proto files.

When it happens

Trigger: protoc exits non-zero during generate_proto_files due to malformed/unsupported proto files, an incompatible protoc version, or missing import paths.

Common situations: A too-old or too-new protoc/protobuf mismatch; corrupted or hand-edited .proto files; missing proto dependencies in the checkout.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8c416c04c9c877ba. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/gen_protos.py:496

      [sys.executable] +  # expecting to be called from command line
      ['--proto_path=%s' % builtin_protos] +
      ['--proto_path=%s' % d
      for d in proto_dirs] + ['--python_out=%s' % PYTHON_OUTPUT_PATH] +
      ['--plugin=protoc-gen-mypy=%s' % protoc_gen_mypy] +
      # new version of mypy-protobuf converts None to zero default value
      # and remove Optional from the param type annotation. This causes
      # some mypy errors. So to mitigate and fall back to old behavior,
      # use `relax_strict_optional_primitives` flag. more at
      # https://github.com/nipunn1313/mypy-protobuf/tree/main#relax_strict_optional_primitives # pylint:disable=line-too-long
      ['--mypy_out=relax_strict_optional_primitives:%s' % PYTHON_OUTPUT_PATH
      ] +
      # TODO(robertwb): Remove the prefix once it's the default.
      ['--grpc_python_out=grpc_2_0:%s' % PYTHON_OUTPUT_PATH] + proto_files)

  LOG.info('Regenerating Python proto definitions (%s).' % regenerate_reason)
  ret_code = protoc.main(args)
  if ret_code:
    raise RuntimeError(
        'Protoc returned non-zero status (see logs for details): '
        '%s' % ret_code)

  # copy resource files
  for path in MODEL_RESOURCES:
    shutil.copy2(os.path.join(PROJECT_ROOT, path), PYTHON_OUTPUT_PATH)

  proto_packages = set()
  # see: https://github.com/protocolbuffers/protobuf/issues/1491
  # force relative import paths for proto files
  compiled_import_re = re.compile('^from (.*) import (.*)$')
  for file_path in find_by_ext(PYTHON_OUTPUT_PATH,
                              ('_pb2.py', '_pb2_grpc.py', '_pb2.pyi')):
    proto_packages.add(os.path.dirname(file_path))
    lines = []
    with open(file_path, encoding='utf-8') as f:
      for line in f:
        match_obj = compiled_import_re.match(line)

View on GitHub (pinned to 12126d8942)