apache/beam · error · ValueError

No MLTransform found. Please install tensorflow-transform or

Error message

No MLTransform found. Please install tensorflow-transform or sentence-transformers to use this transform.

What it means

apache_beam.yaml.ml_transform's ml_transform raises this ValueError when the MLTransform class failed to import, i.e. neither tensorflow-transform nor sentence-transformers is installed in the current Python environment. The YAML ML transform is only a thin wrapper around MLTransform, so without one of these backends there is nothing to construct. The check happens before YamlOptions.check_enabled, so it fires even in pipelines that would otherwise be valid.

Source

Thrown at sdks/python/apache_beam/yaml/yaml_ml.py:607

def _config_to_obj(spec):
  if 'type' not in spec:
    raise ValueError(f"Missing type in ML transform spec {spec}")
  if 'config' not in spec:
    raise ValueError(f"Missing config in ML transform spec {spec}")
  constructor = _transform_constructors.get(spec['type'])
  if constructor is None:
    raise ValueError("Unknown ML transform type: %r" % spec['type'])
  return constructor(**spec['config'])


@beam.ptransform.ptransform_fn
def ml_transform(
    pcoll,
    write_artifact_location: Optional[str] = None,
    read_artifact_location: Optional[str] = None,
    transforms: Optional[list[Any]] = None):
  if MLTransform is None:
    raise ValueError(
        'No MLTransform found. Please install tensorflow-transform or '
        'sentence-transformers to use this transform.')
  options.YamlOptions.check_enabled(pcoll.pipeline, 'ML')
  result_ml_transform = MLTransform(
      write_artifact_location=write_artifact_location,
      read_artifact_location=read_artifact_location,
      transforms=[_config_to_obj(t) for t in transforms] if transforms else [])

  if transforms:
    embedding_transforms = [
        t for t in transforms if t.get('type', '').endswith('Embeddings')
    ]
    if embedding_transforms:
      from apache_beam.typehints import List
      try:
        if pcoll.element_type:
          columns_to_change = {
              col

View on GitHub (pinned to 12126d8942)

Solutions

1. Install a backend: pip install 'apache-beam[yaml]' with tensorflow-transform (pip install tensorflow-transform) or sentence-transformers (pip install sentence-transformers).
2. Verify the import works: python -c "from apache_beam.ml.transforms.base import MLTransform" in the same interpreter/venv that runs the pipeline.
3. If you intended to gate this transform, enable it via the YAML option check (options.YamlOptions.check_enabled(..., 'ML')) only after installing a backend.

Example fix

// before
pipeline:
  - type: ml_transform
    ...
// after
# shell
pip install tensorflow-transform
# then run the same pipeline

import importlib
if importlib.util.find_spec('tensorflow_transform') is None and importlib.util.find_spec('sentence_transformers') is None:
    raise SystemExit('Install tensorflow-transform or sentence-transformers before using ml_transform')

def ml_backend_available() -> bool:
    import importlib.util
    return any(importlib.util.find_spec(m) for m in ('tensorflow_transform', 'sentence_transformers'))

When it happens

Trigger: Calling the 'ml_transform' YAML transform (yaml_ml.py:607 ml_transform) when `from apache_beam.ml.transforms.base import MLTransform` returned None due to ImportError of both optional backends. Any pipeline spec containing type: ml_transform run in an environment without the ML extras.

Common situations: Running a Beam YAML pipeline that preprocesses data for ML inference on a machine where only apache-beam core was pip-installed; CI containers with slim Beam images; switching Python venvs where the ML extras were installed elsewhere.

Understand the failure class

Background: "X is not installed. Please install it with pip install Y": missing optional dependency errors — ImportError/ValueError raised when a library's optional extra was never installed — this error's family across 22 libraries.

Related errors

• Invalid model_handler specification. Expected dict but was {
• Unexpected parameters in model_handler: {extra_params}
• Missing parameters in model_handler: {missing_params}
• Unknown model handler type: {typ}.
• Missing type in ML transform spec {spec}

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a1e9c88ad6c45ac6. Report an issue: GitHub.