apache/beam · error · Exception

Cannot override RemoteModelHandler.load_model, implement cre

Error message

Cannot override RemoteModelHandler.load_model, implement create_client instead.

What it means

RemoteModelHandler subclasses in Beam's ML inference framework must not override load_model(). The shared client lifecycle (creating one client per worker via shared.Shared) is managed by the base class; subclasses customize only how the client is created (create_client) and how requests are made (request).

Source

Thrown at sdks/python/apache_beam/ml/inference/base.py:461

      rate_limiter: A RateLimiter object for setting a global rate limit.
    """
    # Configure ReactiveThrottler for client-side throttling behavior.
    self.throttler = ReactiveThrottler(
        window_ms=window_ms,
        bucket_ms=bucket_ms,
        overload_ratio=overload_ratio,
        namespace=namespace,
        throttle_delay_secs=throttle_delay_secs)
    self.logger = logging.getLogger(namespace)
    self.num_retries = num_retries
    self.retry_filter = retry_filter
    self._rate_limiter = rate_limiter
    self._shared_rate_limiter = None
    self._shared_handle = shared.Shared()

  def __init_subclass__(cls):
    if cls.load_model is not RemoteModelHandler.load_model:
      raise Exception(
          "Cannot override RemoteModelHandler.load_model, ",
          "implement create_client instead.")
    if cls.run_inference is not RemoteModelHandler.run_inference:
      raise Exception(
          "Cannot override RemoteModelHandler.run_inference, ",
          "implement request instead.")

  @abstractmethod
  def create_client(self) -> ModelT:
    """Creates the client that is used to make the remote inference request
    in request(). All relevant arguments should be passed to __init__().
    """
    raise NotImplementedError(type(self))

  def load_model(self) -> ModelT:
    return self.create_client()

  def retry_on_exception(func):

View on GitHub (pinned to 12126d8942)

Solutions

1. Delete the load_model override from your RemoteModelHandler subclass.
2. Implement create_client() to construct and return the inference client (move any client construction from load_model there).
3. Implement request() for the per-request inference call; batching/retry/rate limiting is handled by the base class.

Example fix

# before
class MyHandler(RemoteModelHandler):
  def load_model(self):
    return openai.Client(api_key=self._api_key)

# after
class MyHandler(RemoteModelHandler):
  def create_client(self):
    return openai.Client(api_key=self._api_key)
  def request(self, batch, client, inference_args):
    ...

assert not 'load_model' in MyHandler.__dict__, 'RemoteModelHandler subclasses must implement create_client, not load_model'

def valid_remote_handler(cls):
    return issubclass(cls, RemoteModelHandler) and cls.load_model is RemoteModelHandler.load_model

try:
    handler = MyHandler(...)
except Exception as e:
    if 'Cannot override RemoteModelHandler.load_model' in str(e):
        raise TypeError('Remove load_model; implement create_client()') from e
    raise

When it happens

Trigger: Defining a load_model method on a subclass of RemoteModelHandler (e.g. an OpenAI or MTalkGPT handler). At class definition time, __init_subclass__ compares cls.load_model to RemoteModelHandler.load_model and raises immediately — no instance is needed.

Common situations: Migrating custom handlers written against the older ModelHandler API (where overriding load_model was the norm) to the remote-handler API; copy-pasting a local ModelHandler implementation and turning it into a remote one; IDE autogeneration of 'abstract' method stubs including load_model.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Cannot override RemoteModelHandler.run_inference, implement
• Rate Limit Exceeded, Could not process this batch.
• Cannot make make an unkeyed model handler with pre or postpr
• Cannot use an unkeyed model handler with pre or postprocessi
• Empty list maps to model handler {mh}. All model handlers mu

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/addc175e415f36d9. Report an issue: GitHub.