apache/beam · error · TypeError

Unknown proto implementation

Error message

Unknown proto implementation: {api_implementation.Type()}

What it means

gen_protos.py selects the protobuf runtime implementation (upb, cpp, or python) to pick the right repeated-field container types. If api_implementation.Type() returns an unrecognized value, generation cannot proceed and raises TypeError.

Solutions

1. Pin protobuf to a supported version (e.g. pip install 'protobuf<5' or a version known to work with Beam).
2. Force the pure-python implementation via PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python.
3. Update gen_protos.py to handle the new implementation type if supporting a newer protobuf.

Example fix

// before (shell)
pip install -U protobuf
// after
pip install 'protobuf==4.25.3'

from google.protobuf import internal
impl = internal.api_implementation.Type()
assert impl in ('upb', 'cpp', 'python'), f"unsupported protobuf impl: {impl}"

def has_supported_protobuf_impl():
    from google.protobuf import internal
    return internal.api_implementation.Type() in ('upb', 'cpp', 'python')

try:
    generate_proto_files()
except TypeError as e:
    if 'Unknown proto implementation' in str(e):
        os.environ['PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION'] = 'python'
        # re-exec or reinstall a supported protobuf

When it happens

Trigger: Running the proto generation script with a protobuf runtime whose api_implementation.Type() is not one of the handled values (e.g. a very new or unusual protobuf build).

Common situations: Installing an experimental or future protobuf version with a new implementation mode; running in an environment with a custom protobuf build.

Understand the failure class

Background: "unsupported platform" / "not supported on this platform" errors: what they mean and how to fix them — this error's family across 47 libraries.

Related errors

• Cannot convert from nanoseconds to microseconds because…
• cannot encode a null ByteString
• Cannot interpret a request received over control channel…
• Cannot provide because is not a subclass of
• Could not find enum descriptor

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/86629a266205d40a. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/gen_protos.py:144

    from google.protobuf.internal import containers
    repeated_types = (
        list,
        containers.RepeatedScalarFieldContainer,
        containers.RepeatedCompositeFieldContainer)
  elif api_implementation.Type() == 'upb':
    from google._upb import _message
    repeated_types = (
        list,
        _message.RepeatedScalarContainer,
        _message.RepeatedCompositeContainer)
  elif api_implementation.Type() == 'cpp':
    from google.protobuf.pyext import _message
    repeated_types = (
        list,
        _message.RepeatedScalarContainer,
        _message.RepeatedCompositeContainer)
  else:
    raise TypeError(
        "Unknown proto implementation: " + api_implementation.Type())

  class Context(object):
    INDENT = '  '
    CAP_SPLIT = re.compile('([A-Z][^A-Z]*|^[a-z]+)')

    def __init__(self, indent=0):
      self.lines = []
      self.imports = set()
      self.empty_types = set()
      self._indent = indent

    @contextlib.contextmanager
    def indent(self):
      self._indent += 1
      yield
      self._indent -= 1

View on GitHub (pinned to 12126d8942)