apache/beam · error · ValueError

Invalid model_handler specification.

Error message

Invalid model_handler specification.

What it means

`ModelHandlerProvider.parse_processing_transform` (yaml_ml.py:108) expects the model handler specification to be a dict that names a registered handler type. If the spec is provided but is not a dict, ValueError('Invalid model_handler specification.') is raised.

Solutions

1. Wrap the handler spec in a mapping with the appropriate type key, e.g. {type: VertexAIModelHandlerJSON, project: ..., endpoint_id: ...}.
2. Check that the YAML indentation keeps model_handler as a nested dict, not a scalar.
3. Verify the handler type string is registered via ModelHandlerProvider.register_handler_type.

Example fix

# before
model_handler: VertexAIModelHandlerJSON
# after
model_handler:
  type: VertexAIModelHandlerJSON
  project: my-project
  endpoint_id: 123

spec = cfg.get('model_handler')
if spec is not None and not isinstance(spec, dict):
    raise ValueError('model_handler must be a mapping with a type key')

def is_handler_spec(v):
    return isinstance(v, dict) and isinstance(v.get('type'), str)

try:
    handler = ModelHandlerProvider.parse_processing_transform(spec, typ)
except ValueError as e:
    raise YamlConfigError('model_handler must be a dict like {type: ...}') from e

When it happens

Trigger: Passing the model_handler / processing transform spec as a string or other non-dict value, e.g. model_handler: VertexAIModelHandlerJSON instead of model_handler: {type: VertexAIModelHandlerJSON, ...}.

Common situations: YAML config where the handler is written as a plain scalar name instead of a mapping with a type key; JSON configs converted incorrectly; copy-paste from docs losing nesting.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• Cannot specify 'callable' with 'path' and 'name' for
• Model Handler does not implement a default preprocess…
• Must specify one of 'callable' or 'path' and 'name' for
• tests attribute must be a list of test specifications.
• Unable to instantiate model handler of type

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/200bf64cc8b6231f. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/yaml_ml.py:108

      if callable and (path or name):
        raise ValueError(
            f"Cannot specify 'callable' with 'path' and 'name' for {typ} "
            f"function.")
      if path and name:
        return python_callable.PythonCallableWithSource.load_from_script(
            FileSystems.open(path).read().decode(), name)
      elif callable:
        return python_callable.PythonCallableWithSource(callable)
      else:
        raise ValueError(
            f"Must specify one of 'callable' or 'path' and 'name' for {typ} "
            f"function.")

    if processing_transform:
      if isinstance(processing_transform, dict):
        return _parse_config(**processing_transform)
      else:
        raise ValueError("Invalid model_handler specification.")

  def underlying_handler(self):
    return self._handler

  @staticmethod
  def default_preprocess_fn():
    raise ValueError(
        'Model Handler does not implement a default preprocess '
        'method. Please define a preprocessing method using the '
        '\'preprocess\' tag. This is required in most cases because '
        'most models will have a different input shape, so the model '
        'cannot generalize how the input Row should be transformed. For '
        'an example preprocess method, see VertexAIModelHandlerJSONProvider')

  def _preprocess_fn_internal(self):
    return lambda row: (row, self._preprocess_fn(row))

  @staticmethod

View on GitHub (pinned to 12126d8942)