What causes "No source nodes passed evaluation." in run-llama/llama_index?

Improve retrieval quality: check index contents, embedding model, top_k, and chunking so relevant nodes are actually retrieved

run-llama/llama_index · error · ValueError

No source nodes passed evaluation.

Error message

No source nodes passed evaluation.

What it means

RetrySourceQueryEngine evaluates each retrieved source node with a relevance evaluator and keeps only nodes that pass. If every node fails evaluation, there is nothing left to build a refined SummaryIndex from, so it raises ValueError('No source nodes passed evaluation.') before attempting another retry round.

Source

Thrown at llama-index-core/llama_index/core/query_engine/retry_source_query_engine.py:77

        else:
            logger.debug("Evaluation returned False.")
            # Test source nodes
            source_evals = [
                self._evaluator.evaluate(
                    query=query_str,
                    response=typed_response.response,
                    contexts=[source_node.get_content()],
                )
                for source_node in typed_response.source_nodes
            ]
            orig_nodes = typed_response.source_nodes
            assert len(source_evals) == len(orig_nodes)
            new_docs = []
            for node, eval_result in zip(orig_nodes, source_evals):
                if eval_result:
                    new_docs.append(Document(text=node.node.get_content()))
            if len(new_docs) == 0:
                raise ValueError("No source nodes passed evaluation.")
            new_index = SummaryIndex.from_documents(
                new_docs,
            )
            new_retriever_engine = RetrieverQueryEngine(new_index.as_retriever())
            new_query_engine = RetrySourceQueryEngine(
                new_retriever_engine,
                self._evaluator,
                self._llm,
                self.max_retries - 1,
            )
            return new_query_engine.query(query_bundle)

    async def _aquery(self, query_bundle: QueryBundle) -> RESPONSE_TYPE:
        """Not supported."""
        return self._query(query_bundle)

View on GitHub (pinned to afd0fef371)

Solutions

Improve retrieval quality: check index contents, embedding model, top_k, and chunking so relevant nodes are actually retrieved
Lower evaluator strictness (e.g. a more lenient LLM or custom evaluator prompt) so borderline-relevant nodes pass
Wrap the query call in try/except ValueError and surface a 'no relevant sources' answer to the user instead of crashing

Example fix

// before
response = retry_engine.query("What is the refund policy?")

// after
try:
    response = retry_engine.query("What is the refund policy?")
except ValueError as e:
    if "No source nodes passed evaluation" in str(e):
        response = Response("No relevant sources were found for this question.")
    else:
        raise

Defensive patterns

Strategy: try-catch

Validate before calling

# Pre-check: evaluate retrieval yourself is costly; instead validate corpus coverage
# cheap sanity check that the index is non-empty and top-k retrieval returns nodes
nodes = retriever_query_engine.retrieve(QueryBundle(query_str=q))
if not nodes:
    raise RuntimeError("Retriever returned nothing; fix index before using RetrySourceQueryEngine")

Try / catch

from llama_index.core.response.schema import Response

try:
    resp = retry_engine.query(q)
except ValueError as e:
    if "No source nodes passed evaluation" in str(e):
        resp = Response(
            "No relevant sources were found for this question.",
            source_nodes=[],
        )
    else:
        raise

Prevention

Verify index quality with a few probe queries before adding retry/evaluator layers
Keep max_retries small (1-2) and always catch ValueError as the terminal 'retries exhausted' signal
Tune evaluator prompt/LLM strictness if legitimate sources are being rejected

When it happens

Trigger: Constructing RetrySourceQueryEngine(retriever_query_engine, evaluator, llm, max_retries=N) and calling .query() where the evaluator (e.g. RelevancyEvaluator) judges all retrieved source nodes as irrelevant to the query.

Common situations: Retriever returns off-topic chunks (poor index quality, wrong embedding model, chunking mismatch), an overly strict relevancy evaluator/LLM, or a query that genuinely has no answer in the indexed corpus.

Related errors

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/a685f338fab5a417. Report an issue: GitHub.