run-llama/llama_index · error · ValueError

Cannot specify both similarity_fn and similarity_mode

Error message

Cannot specify both similarity_fn and similarity_mode

What it means

Raised by SemanticSimilarityEvaluator.__init__ when the caller passes both a custom similarity_fn (a callable over two embeddings) and a similarity_mode (an enum selecting a built-in mode). The two parameters are alternative ways to configure scoring — when similarity_fn is given, similarity_mode is ignored-by-design, so supplying both is rejected as an ambiguous configuration.

Source

Thrown at llama-index-core/llama_index/core/evaluation/semantic_similarity.py:45

            Defaults to 0.8.

    """

    def __init__(
        self,
        embed_model: Optional[BaseEmbedding] = None,
        similarity_fn: Optional[Callable[..., float]] = None,
        similarity_mode: Optional[SimilarityMode] = None,
        similarity_threshold: float = 0.8,
    ) -> None:
        self._embed_model = embed_model or Settings.embed_model

        if similarity_fn is None:
            similarity_mode = similarity_mode or SimilarityMode.DEFAULT
            self._similarity_fn = lambda x, y: similarity(x, y, mode=similarity_mode)
        else:
            if similarity_mode is not None:
                raise ValueError(
                    "Cannot specify both similarity_fn and similarity_mode"
                )
            self._similarity_fn = similarity_fn

        self._similarity_threshold = similarity_threshold

    def _get_prompts(self) -> PromptDictType:
        """Get prompts."""
        return {}

    def _update_prompts(self, prompts: PromptDictType) -> None:
        """Update prompts."""

    async def aevaluate(
        self,
        query: Optional[str] = None,
        response: Optional[str] = None,
        contexts: Optional[Sequence[str]] = None,

View on GitHub (pinned to afd0fef371)

Solutions

1. Pass only similarity_fn if you have a custom callable: SemanticSimilarityEvaluator(similarity_fn=cos_sim_fn).
2. Or pass only similarity_mode to use the built-in similarity() helper: SemanticSimilarityEvaluator(similarity_mode=SimilarityMode.DEFAULT).
3. Strip the redundant key from config dicts before constructing the evaluator.

Example fix

# before
evaluator = SemanticSimilarityEvaluator(
    similarity_fn=cosine_sim,
    similarity_mode=SimilarityMode.COSINE,  # both set -> ValueError
)

# after
evaluator = SemanticSimilarityEvaluator(similarity_fn=cosine_sim)

if similarity_fn is not None and similarity_mode is not None:
    raise ValueError("Configure similarity_fn OR similarity_mode, not both")
evaluator = SemanticSimilarityEvaluator(
    similarity_fn=similarity_fn, similarity_mode=similarity_mode
)

When it happens

Trigger: Constructing SemanticSimilarityEvaluator(similarity_fn=my_cosine, similarity_mode=SimilarityMode.COSINE) or any call that sets both kwargs while tuning how response/reference embeddings are compared.

Common situations: Copy-pasting an evaluator config that used similarity_mode and then adding a custom fn (or vice versa); upgrading code that previously tolerated both because the mode was silently ignored; LLM-generated configs setting every available option.

Related errors

• Invalid metric name: {metric}
• query and response must be provided
• query, contexts, and response must be provided
• Metric key {metric_key} not in results_df
• names and results_arr must have same length.

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/3a8ea8e14a400b7f. Report an issue: GitHub.