{"record":{"id":"bf3ce2db9f525b3e","repo":"apache/beam","slug":"model-handler-does-not-implement-a-default-preprocess-method","errorCode":null,"errorMessage":"Model Handler does not implement a default preprocess method. Please define a preprocessing method using the 'preprocess' tag. This is required in most cases because most models will have a different input shape, so the model cannot generalize how the input Row should be transformed. For an example preprocess method, see VertexAIModelHandlerJSONProvider","messagePattern":"Model Handler does not implement a default preprocess method\\. Please define a preprocessing method using the 'preprocess' tag\\. This is required in most cases because most models will have a different input shape, so the model cannot generalize how the input Row should be transformed\\. For an example preprocess method, see VertexAIModelHandlerJSONProvider","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/yaml/yaml_ml.py","lineNumber":115,"sourceCode":"      elif callable:\n        return python_callable.PythonCallableWithSource(callable)\n      else:\n        raise ValueError(\n            f\"Must specify one of 'callable' or 'path' and 'name' for {typ} \"\n            f\"function.\")\n\n    if processing_transform:\n      if isinstance(processing_transform, dict):\n        return _parse_config(**processing_transform)\n      else:\n        raise ValueError(\"Invalid model_handler specification.\")\n\n  def underlying_handler(self):\n    return self._handler\n\n  @staticmethod\n  def default_preprocess_fn():\n    raise ValueError(\n        'Model Handler does not implement a default preprocess '\n        'method. Please define a preprocessing method using the '\n        '\\'preprocess\\' tag. This is required in most cases because '\n        'most models will have a different input shape, so the model '\n        'cannot generalize how the input Row should be transformed. For '\n        'an example preprocess method, see VertexAIModelHandlerJSONProvider')\n\n  def _preprocess_fn_internal(self):\n    return lambda row: (row, self._preprocess_fn(row))\n\n  @staticmethod\n  def default_postprocess_fn():\n    return lambda x: x\n\n  def _postprocess_fn_internal(self):\n    return lambda result: (result[0], self._postprocess_fn(result[1]))\n\n  @staticmethod","sourceCodeStart":97,"sourceCodeEnd":133,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/yaml/yaml_ml.py#L97-L133","documentation":"Beam YAML requires a preprocessing function for ML transforms because model input shapes vary. `default_preprocess_fn` (yaml_ml.py:115) is a placeholder that always raises ValueError telling the user to define a `preprocess` config; it is invoked when no custom preprocess function was supplied for the model handler.","triggerScenarios":"Running an ML transform (e.g. RunInference via YAML) on a model handler that lacks a built-in default preprocess, without specifying a `preprocess` callable/path in the transform config.","commonSituations":"First-time users wiring a custom model handler without implementing a default preprocess; omitting the preprocess block because examples with VertexAIModelHandlerJSONProvider (which has a default) worked before.","solutions":["Add a `preprocess` spec to the transform: {path: preprocess.py, name: my_preprocess_fn} or an inline `callable`.","Implement a default_preprocess_fn on a custom ModelHandlerProvider so YAML users get sensible defaults.","Use VertexAIModelHandlerJSONProvider (or another handler with a built-in default preprocess) if no custom preprocessing is needed."],"exampleFix":"# before\n- type: RunInference\n  model_handler: {type: MyHandler, ...}\n# after\n- type: RunInference\n  model_handler: {type: MyHandler, ...}\n  preprocess: {path: preprocess.py, name: preprocess_fn}","handlingStrategy":"fallback","validationCode":"handler_provider = ModelHandlerProvider.handler_types.get(spec['type'])\nif handler_provider and handler_provider.default_preprocess_fn.__qualname__ == 'default_preprocess_fn':\n    assert 'preprocess' in cfg, 'this handler has no default preprocess; define one'","typeGuard":"def needs_explicit_preprocess(provider_cls):\n    import inspect\n    src = inspect.getsource(provider_cls.default_preprocess_fn)\n    return 'raise' in src  # placeholder that raises","tryCatchPattern":"try:\n    transform = RunInferenceYamlTransform(cfg)\nexcept ValueError as e:\n    if 'default preprocess' in str(e):\n        cfg['preprocess'] = {'path': 'preprocess.py', 'name': 'preprocess_fn'}\n        transform = RunInferenceYamlTransform(cfg)\n    else:\n        raise","preventionTips":["Always define a preprocess step for custom model handlers in YAML pipelines.","Implement default_preprocess_fn on custom providers when a sensible default exists.","Prefer handlers with built-in defaults (e.g. VertexAIModelHandlerJSONProvider) for zero-config use."],"tags":["python","apache-beam","yaml-ml","preprocessing"],"backgroundTag":"missing-required-config","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}