{"record":{"id":"d8dc29ac81b5c4ad","repo":"apache/beam","slug":"cannot-interpret-a-request-received-over-control-channel","errorCode":null,"errorMessage":"Cannot interpret a request received over control channel. This is not expected. Verify that SDK was not accidentally downgraded at runtime. SDK version: {beam_version}, instruction id: {work_request.instruction_id}, raw request: {str(work_request.SerializeToString())}","messagePattern":"Cannot interpret a request received over control channel\\. This is not expected\\. Verify that SDK was not accidentally downgraded at runtime\\. SDK version: (.+?), instruction id: (.+?), raw request: (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"sdks/python/apache_beam/runners/worker/sdk_worker.py","lineNumber":270,"sourceCode":"    no_more_work = Sentinel.sentinel\n\n    def get_responses():\n      # type: () -> Iterator[beam_fn_api_pb2.InstructionResponse]\n      while True:\n        response = self._responses.get()\n        if response is no_more_work:\n          return\n        yield response\n\n    self._alive = True\n\n    try:\n      for work_request in self._control_stub.Control(get_responses()):\n        _LOGGER.debug('Got work %s', work_request.instruction_id)\n        request_type = work_request.WhichOneof('request')\n\n        if request_type is None:\n          raise RuntimeError(\n              \"Cannot interpret a request received over control channel. \"\n              \"This is not expected. \"\n              \"Verify that SDK was not accidentally downgraded at runtime. \"\n              f\"SDK version: {beam_version}, \"\n              f\"instruction id: {work_request.instruction_id}, \"\n              f\"raw request: {str(work_request.SerializeToString())}\")\n\n        # Name spacing the request method with 'request_'. The called method\n        # will be like self.request_register(request)\n        getattr(self, SdkHarness.REQUEST_METHOD_PREFIX + request_type)(\n            work_request)\n    finally:\n      self._alive = False\n      if self.data_sampler:\n        self.data_sampler.stop()\n\n    _LOGGER.info('No more requests from control plane')\n    _LOGGER.info('SDK Harness waiting for in-flight requests to complete')","sourceCodeStart":252,"sourceCodeEnd":288,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/worker/sdk_worker.py#L252-L288","documentation":" SdkHarness.run iterates the control channel stream and dispatches requests by WhichOneof('request'). A request with no set oneof field cannot be interpreted, so it raises RuntimeError advising the SDK may have been downgraded at runtime, and includes the SDK version, instruction id, and raw serialized request for diagnosis.","triggerScenarios":" The runner (or job server) sends a control request whose 'request' oneof is unset — typically a newer runner emitting request types an older SDK protobuf cannot decode, producing an empty message.","commonSituations":" Beam version skew: runner/Flink/Spark job server newer than the SDK harness image; a stale container image with an older apache-beam; heterogeneous worker images after a pipeline upgrade.","solutions":["Pin the SDK harness container image to the exact same Beam version as the runner/job server.","Rebuild/re-push the worker image and clear cached images so the new version is actually pulled.","Check the container environment for pip installs that downgrade apache-beam at startup (e.g. requirements.txt pinning an older beam).","Use --environment_config/--worker_image (or runner equivalent) to force the matching image, and redeploy."],"exampleFix":"// before\n# Dockerfile worker image\nFROM apache/beam_python3.11_sdk:2.50.0\n// after\n# match the runner, e.g. DataflowBeam 2.58.0\nFROM apache/beam_python3.11_sdk:2.58.0","handlingStrategy":"try-catch","validationCode":"# fail fast at deploy time if versions diverge\nassert sdk_harness_beam_version == runner_beam_version, \\\n    f'SDK harness {sdk_harness_beam_version} != runner {runner_beam_version}'","typeGuard":null,"tryCatchPattern":"try:\n    harness.run()\nexcept RuntimeError as e:\n    if 'Cannot interpret a request' in str(e):\n        log.critical('SDK/runner version skew detected: %s', e)\n        # redeploy with matched image; do not blind-retry\n    raise","preventionTips":["Pin the worker container image tag to the exact runner Beam version.","Block apache-beam downgrades in worker requirements.txt.","Verify the image digest actually deployed (avoid stale image caches)."],"tags":["apache-beam","grpc","version-skew","protobuf"],"backgroundTag":"protobuf-unmarshal-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}