apache/beam · error · ImportError

sentence-transformers is required to use HuggingfaceTextEmbe

Error message

sentence-transformers is required to use HuggingfaceTextEmbeddings.Please install it with using `pip install sentence-transformers`.

What it means

HuggingfaceTextEmbeddings.__init__ raises this ImportError when the sentence-transformers package is not installed. The module imports SentenceTransformer in a guarded try/except (leaving it None), and the constructor fails fast with a pip install hint. This class embeds text via SentenceTransformer models, which cannot function without the library.

Source

Thrown at sdks/python/apache_beam/ml/rag/embeddings/huggingface.py:70

        model_name: Name of the sentence-transformers model to use.
        max_seq_length: Maximum sequence length for the model.
        **kwargs: Additional arguments passed to
            :class:`~apache_beam.ml.transforms.base.EmbeddingsManager`,
            including:

            - ``load_model_args``: dict passed to
              ``SentenceTransformer()`` constructor
              (e.g. ``device``, ``cache_folder``).
            - ``min_batch_size`` / ``max_batch_size``:
              Control batching for inference.
            - ``large_model``: If True, share the model
              across processes to reduce memory usage.
            - ``inference_args``: dict passed to
              ``model.encode()``
              (e.g. ``normalize_embeddings``).
    """
    if not SentenceTransformer:
      raise ImportError(
          "sentence-transformers is required to use "
          "HuggingfaceTextEmbeddings."
          "Please install it with using `pip install sentence-transformers`.")
    super().__init__(type_adapter=create_text_adapter(), **kwargs)
    self.model_name = model_name
    self.max_seq_length = max_seq_length
    self.model_class = SentenceTransformer

  def get_model_handler(self):
    """Returns model handler configured with RAG adapter."""
    return _SentenceTransformerModelHandler(
        model_class=self.model_class,
        max_seq_length=self.max_seq_length,
        model_name=self.model_name,
        load_model_args=self.load_model_args,
        min_batch_size=self.min_batch_size,
        max_batch_size=self.max_batch_size,
        large_model=self.large_model)

View on GitHub (pinned to 12126d8942)

Solutions

Install the dependency: pip install sentence-transformers.
Add sentence-transformers to your requirements file passed to the runner (--requirements_file for Dataflow).
Use a different embeddings manager (e.g. VertexAITextEmbeddings) if you cannot install the package.

Example fix

// before
embedder = HuggingfaceTextEmbeddings(model_name='sentence-transformers/all-MiniLM-L6-v2')

// after
# terminal: pip install sentence-transformers
embedder = HuggingfaceTextEmbeddings(model_name='sentence-transformers/all-MiniLM-L6-v2')

Defensive patterns

Strategy: try-catch

Validate before calling

import importlib.util

def sentence_transformers_available() -> bool:
    return importlib.util.find_spec('sentence_transformers') is not None

Try / catch

try:
    embedder = HuggingfaceTextEmbeddings(model_name='sentence-transformers/all-MiniLM-L6-v2')
except ImportError:
    logging.error('Install: pip install sentence-transformers')
    raise

Prevention

Pin sentence-transformers in the requirements file sent to your runner.
Verify worker environments with a startup import check or pip check in CI.
Prefer full ML base images over slim ones for embedding pipelines.

When it happens

Trigger: Constructing HuggingfaceTextEmbeddings(model_name=...) in an environment where `pip install sentence-transformers` was never run; Beam Dataflow/Flink workers missing the package because the requirements file didn't include it.

Common situations: Fresh venv or CI container with only apache_beam installed; forgetting to ship extra dependencies to remote runners; a slim Docker image that trimmed ML dependencies.

Understand the failure class

Background: "X is not installed. Please install it with pip install Y": missing optional dependency errors — ImportError/ValueError raised when a library's optional extra was never installed — this error's family across 22 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9b2f109f10cbdf76. Report an issue: GitHub.