apache/beam · error · Exception

Cannot override RemoteModelHandler.run_inference, implement

Error message

Cannot override RemoteModelHandler.run_inference, implement request instead.

What it means

RemoteModelHandler subclasses must not override run_inference(). The base class's run_inference handles batching, throttling, retries, and rate limiting, then delegates a single batch to request(). Customizing the network call is done by overriding request instead.

Source

Thrown at sdks/python/apache_beam/ml/inference/base.py:465

        window_ms=window_ms,
        bucket_ms=bucket_ms,
        overload_ratio=overload_ratio,
        namespace=namespace,
        throttle_delay_secs=throttle_delay_secs)
    self.logger = logging.getLogger(namespace)
    self.num_retries = num_retries
    self.retry_filter = retry_filter
    self._rate_limiter = rate_limiter
    self._shared_rate_limiter = None
    self._shared_handle = shared.Shared()

  def __init_subclass__(cls):
    if cls.load_model is not RemoteModelHandler.load_model:
      raise Exception(
          "Cannot override RemoteModelHandler.load_model, ",
          "implement create_client instead.")
    if cls.run_inference is not RemoteModelHandler.run_inference:
      raise Exception(
          "Cannot override RemoteModelHandler.run_inference, ",
          "implement request instead.")

  @abstractmethod
  def create_client(self) -> ModelT:
    """Creates the client that is used to make the remote inference request
    in request(). All relevant arguments should be passed to __init__().
    """
    raise NotImplementedError(type(self))

  def load_model(self) -> ModelT:
    return self.create_client()

  def retry_on_exception(func):
    @functools.wraps(func)
    def wrapper(self, *args, **kwargs):
      return retry.with_exponential_backoff(
          num_retries=self.num_retries,

View on GitHub (pinned to 12126d8942)

Solutions

1. Remove the run_inference override from the subclass.
2. Implement request(self, batch, client, inference_args) that performs the actual remote call for one batch.
3. Rely on the base class for batching, retry, throttling, and rate limiting.

Example fix

# before
class MyHandler(RemoteModelHandler):
  def run_inference(self, batch, model, inference_args=None):
    return [model.embed(text) for text in batch]

# after
class MyHandler(RemoteModelHandler):
  def request(self, batch, client, inference_args):
    return client.embed(input=list(batch)).embeddings

assert MyHandler.run_inference is RemoteModelHandler.run_inference, 'Use request(), not run_inference, for remote handlers'

def uses_base_run_inference(cls):
    return cls.run_inference is RemoteModelHandler.run_inference

try:
    handler = MyHandler(...)
except Exception as e:
    if 'Cannot override RemoteModelHandler.run_inference' in str(e):
        raise TypeError('Move inference logic into request()') from e
    raise

When it happens

Trigger: Defining a run_inference method on a subclass of RemoteModelHandler. Detected in __init_subclass__ at class-definition time by comparing cls.run_inference with RemoteModelHandler.run_inference.

Common situations: Porting an existing local ModelHandler whose run_inference was overridden; wanting custom batching logic and copying the base implementation; older tutorials predating the RemoteModelHandler request()/create_client() API.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Cannot override RemoteModelHandler.load_model, implement cre
• Rate Limit Exceeded, Could not process this batch.
• Cannot make make an unkeyed model handler with pre or postpr
• Cannot use an unkeyed model handler with pre or postprocessi
• Empty list maps to model handler {mh}. All model handlers mu

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a7fc086fb7b53c89. Report an issue: GitHub.