headroomlabs-ai/headroom · error · ImportError

sentence-transformers required for semantic similarity. Inst

Error message

sentence-transformers required for semantic similarity. Install with: pip install sentence-transformers

What it means

Error "sentence-transformers required for semantic similarity. Install with: pip install sentence-transformers" thrown in headroomlabs-ai/headroom.

Source

Thrown at headroom/evals/metrics.py:179

    response_b: str,
    model_name: str | None = None,
) -> float:
    """Compute semantic similarity using sentence embeddings.

    Requires sentence-transformers package.

    Args:
        response_a: First response
        response_b: Second response
        model_name: Sentence transformer model to use. Uses config default if None.

    Returns:
        Cosine similarity between embeddings (0.0 to 1.0)
    """
    try:
        import numpy as np
    except ImportError as e:
        raise ImportError(
            "sentence-transformers required for semantic similarity. "
            "Install with: pip install sentence-transformers"
        ) from e

    # Use centralized registry for shared model instances
    from headroom.models.ml_models import MLModelRegistry

    model = MLModelRegistry.get_sentence_transformer(model_name)

    embeddings = model.encode([response_a, response_b])
    embedding_a, embedding_b = embeddings[0], embeddings[1]

    # Cosine similarity
    dot_product = np.dot(embedding_a, embedding_b)
    norm_a = np.linalg.norm(embedding_a)
    norm_b = np.linalg.norm(embedding_b)

    if norm_a == 0 or norm_b == 0:

View on GitHub (pinned to 322425c43b)

Solutions

1. Install sentence-transformers: pip install sentence-transformers
2. Or install the relevance/evals extra that pulls it in
3. If install fails on your platform, use a non-semantic similarity metric instead

When it happens

Trigger: Raised when a semantic-similarity eval metric is requested but the `sentence-transformers` package is not installed.

Common situations: See trigger scenarios.

AI-assisted analysis of headroomlabs-ai/headroom@322425c43b (2026-08-15). Data as JSON: /api/errors/bc4e742459346df3. Report an issue: GitHub.