{"record":{"id":"a7fc086fb7b53c89","repo":"apache/beam","slug":"cannot-override-remotemodelhandler-run-inference-implement","errorCode":null,"errorMessage":"Cannot override RemoteModelHandler.run_inference, implement request instead.","messagePattern":"Cannot override RemoteModelHandler\\.run_inference, implement request instead\\.","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/ml/inference/base.py","lineNumber":465,"sourceCode":"        window_ms=window_ms,\n        bucket_ms=bucket_ms,\n        overload_ratio=overload_ratio,\n        namespace=namespace,\n        throttle_delay_secs=throttle_delay_secs)\n    self.logger = logging.getLogger(namespace)\n    self.num_retries = num_retries\n    self.retry_filter = retry_filter\n    self._rate_limiter = rate_limiter\n    self._shared_rate_limiter = None\n    self._shared_handle = shared.Shared()\n\n  def __init_subclass__(cls):\n    if cls.load_model is not RemoteModelHandler.load_model:\n      raise Exception(\n          \"Cannot override RemoteModelHandler.load_model, \",\n          \"implement create_client instead.\")\n    if cls.run_inference is not RemoteModelHandler.run_inference:\n      raise Exception(\n          \"Cannot override RemoteModelHandler.run_inference, \",\n          \"implement request instead.\")\n\n  @abstractmethod\n  def create_client(self) -> ModelT:\n    \"\"\"Creates the client that is used to make the remote inference request\n    in request(). All relevant arguments should be passed to __init__().\n    \"\"\"\n    raise NotImplementedError(type(self))\n\n  def load_model(self) -> ModelT:\n    return self.create_client()\n\n  def retry_on_exception(func):\n    @functools.wraps(func)\n    def wrapper(self, *args, **kwargs):\n      return retry.with_exponential_backoff(\n          num_retries=self.num_retries,","sourceCodeStart":447,"sourceCodeEnd":483,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/ml/inference/base.py#L447-L483","documentation":"RemoteModelHandler subclasses must not override run_inference(). The base class's run_inference handles batching, throttling, retries, and rate limiting, then delegates a single batch to request(). Customizing the network call is done by overriding request instead.","triggerScenarios":"Defining a run_inference method on a subclass of RemoteModelHandler. Detected in __init_subclass__ at class-definition time by comparing cls.run_inference with RemoteModelHandler.run_inference.","commonSituations":"Porting an existing local ModelHandler whose run_inference was overridden; wanting custom batching logic and copying the base implementation; older tutorials predating the RemoteModelHandler request()/create_client() API.","solutions":["Remove the run_inference override from the subclass.","Implement request(self, batch, client, inference_args) that performs the actual remote call for one batch.","Rely on the base class for batching, retry, throttling, and rate limiting."],"exampleFix":"# before\nclass MyHandler(RemoteModelHandler):\n  def run_inference(self, batch, model, inference_args=None):\n    return [model.embed(text) for text in batch]\n\n# after\nclass MyHandler(RemoteModelHandler):\n  def request(self, batch, client, inference_args):\n    return client.embed(input=list(batch)).embeddings","handlingStrategy":"validation","validationCode":"assert MyHandler.run_inference is RemoteModelHandler.run_inference, 'Use request(), not run_inference, for remote handlers'","typeGuard":"def uses_base_run_inference(cls):\n    return cls.run_inference is RemoteModelHandler.run_inference","tryCatchPattern":"try:\n    handler = MyHandler(...)\nexcept Exception as e:\n    if 'Cannot override RemoteModelHandler.run_inference' in str(e):\n        raise TypeError('Move inference logic into request()') from e\n    raise","preventionTips":["Put per-request logic in request(); leave batching/retry to the base class.","Search subclass bodies for 'def run_inference' during code review.","Instantiate handlers in tests so __init_subclass__ checks run at import time."],"tags":["python","apache-beam","ml-inference","class-definition","api-migration"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}