apache/beam · error · Exception

Cannot make make an unkeyed model handler with pre or postpr

Error message

Cannot make make an unkeyed model handler with pre or postprocessing functions defined into a keyed model handler. All pre/postprocessing functions must be defined on the outer modelhandler.

What it means

KeyedModelHandler wraps a single unkeyed handler for per-key model updates. Decorations like preprocess/postprocess functions belong on the outer (keyed) handler, not the inner one; the constructor rejects an inner handler that already carries preprocessing functions with a plain Exception.

Source

Thrown at sdks/python/apache_beam/ml/inference/base.py:764

    Args:
      unkeyed: Either (a) an implementation of ModelHandler that does not
        require keys or (b) a list of KeyModelMappings mapping lists of keys to
        unkeyed ModelHandlers.
      max_models_per_worker_hint: A hint to the runner indicating how many
        models can be held in memory at one time per worker process. For
        example, if your worker has 8 GB of memory provisioned and your workers
        take up 1 GB each, you should set this to 7 to allow all models to sit
        in memory with some buffer. For more information about memory management,
        see `Use a keyed `ModelHandler <https://beam.apache.org/documentation/ml/about-ml/#use-a-keyed-modelhandler-object>_`.  # pylint: disable=line-too-long
    """
    self._metrics_collectors: dict[str, _MetricsCollector] = {}
    self._default_metrics_collector: _MetricsCollector = None
    self._metrics_namespace = ''
    self._single_model = not isinstance(unkeyed, list)
    if self._single_model:
      if len(unkeyed.get_preprocess_fns()) or len(
          unkeyed.get_postprocess_fns()):
        raise Exception(
            'Cannot make make an unkeyed model handler with pre or '
            'postprocessing functions defined into a keyed model handler. All '
            'pre/postprocessing functions must be defined on the outer model'
            'handler.')
      self._env_vars = getattr(unkeyed, '_env_vars', {})
      self._unkeyed = unkeyed
      return

    self._max_models_per_worker_hint = max_models_per_worker_hint
    # To maintain an efficient representation, we will map all keys in a given
    # KeyModelMapping to a single id (the first key in the KeyModelMapping
    # list). We will then map that key to a ModelHandler. This will allow us to
    # quickly look up the appropriate ModelHandler for any given key.
    self._id_to_mh_map: dict[str, ModelHandler[ExampleT, PredictionT,
                                               ModelT]] = {}
    self._key_to_id_map: dict[str, str] = {}
    for mh_tuple in unkeyed:
      mh = mh_tuple.mh

View on GitHub (pinned to 12126d8942)

Solutions

1. Pass the bare (undecorated) handler into KeyedModelHandler.
2. Apply .with_preprocess_fns() / .with_postprocess_fns() to the KeyedModelHandler itself, not the inner handler.
3. Reorder the pipeline: construct KeyedModelHandler first, then decorate it.

Example fix

# before
mh = MyHandler().with_preprocess_fns(preprocess)
keyed = KeyedModelHandler(mh)

# after
keyed = KeyedModelHandler(MyHandler()).with_preprocess_fns(preprocess)

if len(unkeyed.get_preprocess_fns()) or len(unkeyed.get_postprocess_fns()):
    raise ValueError('Move pre/postprocess fns to the KeyedModelHandler, not the inner handler')

def is_undecorated(mh):
    return not mh.get_preprocess_fns() and not mh.get_postprocess_fns()

try:
    keyed = KeyedModelHandler(unkeyed)
except Exception as e:
    if 'pre or postprocessing' in str(e):
        keyed = KeyedModelHandler(type(unkeyed)(**inner_args)).with_preprocess_fns(...)
    else:
        raise

When it happens

Trigger: Calling KeyedModelHandler(unkeyed) where unkeyed is a single ModelHandler with preprocessing or postprocessing functions attached (e.g. via unkeyed.with_preprocess_fns(...)).

Common situations: Chaining with_preprocess_fns then wrapping in KeyedModelHandler; refactoring a decorated handler into a keyed one for model updates and forgetting to move the functions.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

• Cannot use an unkeyed model handler with pre or postprocessi
• Cannot override RemoteModelHandler.load_model, implement cre
• Cannot override RemoteModelHandler.run_inference, implement
• Rate Limit Exceeded, Could not process this batch.
• Empty list maps to model handler {mh}. All model handlers mu

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/e302fe6287ef1267. Report an issue: GitHub.