Unsupported document_comparison_field: '{self.document_compa

Error message

Unsupported document_comparison_field: '{self.document_comparison_field}'. Use 'content', 'id', or 'meta.<key>'.

What it means

DocumentMAPEvaluator._get_comparison_value resolves each Document's comparison value from the configured document_comparison_field. It raises ValueError when that field is none of 'content', 'id', or a string starting with 'meta.', i.e. the constructor was given an unsupported field specifier.

Source

Thrown at haystack/components/evaluators/document_map.py:80

        Extract the comparison value from a document based on the configured field.
        """
        if self.document_comparison_field == "content":
            return doc.content
        if self.document_comparison_field == "id":
            return doc.id
        if self.document_comparison_field.startswith("meta."):
            parts = self.document_comparison_field[5:].split(".")
            value = doc.meta
            for part in parts:
                if not isinstance(value, dict) or part not in value:
                    return None
                value = value[part]
            return value
        msg = (
            f"Unsupported document_comparison_field: '{self.document_comparison_field}'. "
            "Use 'content', 'id', or 'meta.<key>'."
        )
        raise ValueError(msg)

    def to_dict(self) -> dict[str, Any]:
        """
        Serializes the component to a dictionary.

        :returns:
            Dictionary with serialized data.
        """
        return default_to_dict(self, document_comparison_field=self.document_comparison_field)

    # Refer to https://www.pinecone.io/learn/offline-evaluation/ for the algorithm.
    @component.output_types(score=float, individual_scores=list[float])
    def run(
        self, ground_truth_documents: list[list[Document]], retrieved_documents: list[list[Document]]
    ) -> dict[str, Any]:
        """
        Run the DocumentMAPEvaluator on the given inputs.

View on GitHub (pinned to e318778c9b)

Solutions

Set document_comparison_field to exactly 'content', 'id', or 'meta.<key>'
For a meta field, prefix the key: `document_comparison_field="meta.file_id"`
Validate the config value at construction time before running pipelines

Example fix

// before
evaluator = DocumentMAPEvaluator(document_comparison_field="file_id")
// after
evaluator = DocumentMAPEvaluator(document_comparison_field="meta.file_id")

Defensive patterns

Strategy: validation

Validate before calling

allowed = lambda v: v in ("content", "id") or v.startswith("meta.")
if not allowed(document_comparison_field):
    raise ValueError(f"Unsupported field: {document_comparison_field}")

Type guard

def is_valid_comparison_field(field: object) -> bool:
    return isinstance(field, str) and (field in ("content", "id") or field.startswith("meta."))

Try / catch

try:
    result = evaluator.run(ground_truth_documents=gt, retrieved_documents=ret)
except ValueError as e:
    if "Unsupported document_comparison_field" in str(e):
        evaluator.document_comparison_field = "content"
        result = evaluator.run(ground_truth_documents=gt, retrieved_documents=ret)
    else:
        raise

Prevention

Validate document_comparison_field at construction time
Remember meta keys need the 'meta.' prefix (e.g. 'meta.file_id')
Watch case sensitivity in YAML configs
Round-trip to_dict/from_dict in tests to catch config typos

When it happens

Trigger: Constructing `DocumentMAPEvaluator(document_comparison_field="text")` or any value not 'content'/'id'/'meta.<key>'; a deserialized YAML with a typo like 'Content' or missing 'meta.' prefix; the error surfaces at run() time when documents are compared.

Common situations: Typos in YAML pipeline configs (case sensitivity, missing 'meta.' prefix); using a bare meta key like 'file_id' instead of 'meta.file_id'; migrating configs from older evaluator versions with different option names.

Related errors

AI-assisted analysis of deepset-ai/haystack@e318778c9b (2026-08-30). Data as JSON: /api/errors/c9635e06e9f73f23. Report an issue: GitHub.