Unsupported document_comparison_field: '{self.document_compa

Error message

Unsupported document_comparison_field: '{self.document_comparison_field}'. Use 'content', 'id', or 'meta.<key>'.

What it means

DocumentMRREvaluator._get_comparison_value resolves each Document's comparison value from document_comparison_field and raises ValueError when the configured field is not 'content', 'id', or 'meta.<key>'. Like the MAP evaluator, an invalid field spec only fails at run() time when documents are compared.

Source

Thrown at haystack/components/evaluators/document_mrr.py:78

        Extract the comparison value from a document based on the configured field.
        """
        if self.document_comparison_field == "content":
            return doc.content
        if self.document_comparison_field == "id":
            return doc.id
        if self.document_comparison_field.startswith("meta."):
            parts = self.document_comparison_field[5:].split(".")
            value = doc.meta
            for part in parts:
                if not isinstance(value, dict) or part not in value:
                    return None
                value = value[part]
            return value
        msg = (
            f"Unsupported document_comparison_field: '{self.document_comparison_field}'. "
            "Use 'content', 'id', or 'meta.<key>'."
        )
        raise ValueError(msg)

    def to_dict(self) -> dict[str, Any]:
        """
        Serializes the component to a dictionary.

        :returns:
            Dictionary with serialized data.
        """
        return default_to_dict(self, document_comparison_field=self.document_comparison_field)

    # Refer to https://www.pinecone.io/learn/offline-evaluation/ for the algorithm.
    @component.output_types(score=float, individual_scores=list[float])
    def run(
        self, ground_truth_documents: list[list[Document]], retrieved_documents: list[list[Document]]
    ) -> dict[str, Any]:
        """
        Run the DocumentMRREvaluator on the given inputs.

View on GitHub (pinned to e318778c9b)

Solutions

Use exactly 'content', 'id', or 'meta.<key>' for document_comparison_field
For meta values, write the full path e.g. `document_comparison_field="meta.source.url"`
Check the value stored after from_dict/from_yaml round-trips

Example fix

// before
DocumentMRREvaluator(document_comparison_field="Meta.title")
// after
DocumentMRREvaluator(document_comparison_field="meta.title")

Defensive patterns

Strategy: validation

Validate before calling

if not (document_comparison_field in ("content", "id") or document_comparison_field.startswith("meta.")):
    raise ValueError("field must be 'content', 'id', or 'meta.<key>'")

Type guard

def is_valid_comparison_field(field: object) -> bool:
    return isinstance(field, str) and (field in ("content", "id") or field.startswith("meta."))

Try / catch

try:
    result = evaluator.run(ground_truth_documents=gt, retrieved_documents=ret)
except ValueError as e:
    if "Unsupported document_comparison_field" in str(e):
        evaluator.document_comparison_field = "content"
        result = evaluator.run(ground_truth_documents=gt, retrieved_documents=ret)
    else:
        raise

Prevention

Use exactly one of 'content', 'id', 'meta.<key>'
Prefix meta keys with 'meta.'
Avoid trailing whitespace in YAML values
Unit-test evaluator construction with your production config

When it happens

Trigger: Constructing `DocumentMRREvaluator(document_comparison_field="contents")`, 'meta' without a key, or any unsupported string; the error is raised from run() via _get_comparison_value.

Common situations: Typos in pipeline YAML; forgetting the 'meta.' prefix for a meta key; assuming custom field names like 'blob' are supported.

Related errors

AI-assisted analysis of deepset-ai/haystack@e318778c9b (2026-08-30). Data as JSON: /api/errors/27966a32efd10606. Report an issue: GitHub.