{"record":{"id":"5bc207edf630c88d","repo":"deepset-ai/haystack","slug":"unsupported-document-comparison-field-self-docu-5bc207","errorCode":null,"errorMessage":"Unsupported document_comparison_field: '{self.document_comparison_field}'. Use 'content', 'id', or 'meta.<key>'.","messagePattern":"Unsupported document_comparison_field: '(.+?)'\\. Use 'content', 'id', or 'meta\\.<key>'\\.","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"haystack/components/evaluators/document_ndcg.py","lineNumber":71,"sourceCode":"        Extract the comparison value from a document based on the configured field.\n        \"\"\"\n        if self.document_comparison_field == \"content\":\n            return doc.content\n        if self.document_comparison_field == \"id\":\n            return doc.id\n        if self.document_comparison_field.startswith(\"meta.\"):\n            parts = self.document_comparison_field[5:].split(\".\")\n            value = doc.meta\n            for part in parts:\n                if not isinstance(value, dict) or part not in value:\n                    return None\n                value = value[part]\n            return value\n        msg = (\n            f\"Unsupported document_comparison_field: '{self.document_comparison_field}'. \"\n            \"Use 'content', 'id', or 'meta.<key>'.\"\n        )\n        raise ValueError(msg)\n\n    def _build_relevance_map(self, gt_docs: list[Document]) -> dict[Any, float]:\n        \"\"\"\n        Map each ground truth comparison value to its relevance score.\n\n        Documents whose comparison value cannot be determined (e.g. missing meta key) are skipped,\n        since they can never be matched during retrieval either. Documents that share a comparison\n        value are collapsed to a single entry keeping the highest relevance, so `calculate_dcg` and\n        `calculate_idcg` credit the same unique relevant set with the same scores.\n        \"\"\"\n        relevant_value_to_score: dict[Any, float] = {}\n        for doc in gt_docs:\n            value = self._get_comparison_value(doc)\n            if value is None:\n                continue\n            relevance = doc.score if doc.score is not None else 1.0\n            relevant_value_to_score[value] = max(relevant_value_to_score.get(value, relevance), relevance)\n        return relevant_value_to_score","sourceCodeStart":53,"sourceCodeEnd":89,"githubUrl":"https://github.com/deepset-ai/haystack/blob/e318778c9bf60a1963e3b5f451359655dd696c30/haystack/components/evaluators/document_ndcg.py#L53-L89","documentation":"DocumentNDCGEvaluator._get_comparison_value resolves document comparison values and raises ValueError when document_comparison_field is not 'content', 'id', or 'meta.<key>'. It is reached from both _build_relevance_map (ground truth docs) and calculate_dcg (retrieved docs) during run().","triggerScenarios":"Constructing DocumentNDCGEvaluator with an unsupported field like 'text', 'title' (without 'meta.'), or an empty string; first run() call then fails while building the relevance map.","commonSituations":"YAML config typos or case mistakes ('Content'); using a bare meta key without the 'meta.' prefix; copying a config from another evaluator with different option values.","solutions":["Set document_comparison_field to 'content', 'id', or 'meta.<key>' exactly","Prefix meta keys: `document_comparison_field=\"meta.score_key\"`","Validate the option in __init__ or at pipeline-load time to fail fast"],"exampleFix":"// before\nDocumentNDCGEvaluator(document_comparison_field=\"content \")  # trailing space\n// after\nDocumentNDCGEvaluator(document_comparison_field=\"content\")","handlingStrategy":"validation","validationCode":"field = evaluator.document_comparison_field\nassert field in (\"content\", \"id\") or field.startswith(\"meta.\")","typeGuard":"def is_valid_comparison_field(field: object) -> bool:\n    return isinstance(field, str) and (field in (\"content\", \"id\") or field.startswith(\"meta.\"))","tryCatchPattern":"try:\n    result = evaluator.run(ground_truth_documents=gt, retrieved_documents=ret)\nexcept ValueError as e:\n    if \"Unsupported document_comparison_field\" in str(e):\n        evaluator.document_comparison_field = \"content\"\n        result = evaluator.run(ground_truth_documents=gt, retrieved_documents=ret)\n    else:\n        raise","preventionTips":["Validate the field specifier before pipelines run","For meta fields write 'meta.<key>' including nested paths like 'meta.source.url'","Check YAML for case errors ('Content' is invalid)","Test from_dict on your serialized pipelines"],"tags":["python","value-error","evaluators","configuration"],"backgroundTag":"invalid-config-option","analyzedSha":"e318778c9bf60a1963e3b5f451359655dd696c30","analyzedAt":"2026-08-30T11:45:20.711Z","schemaVersion":2},"datasetVersion":"2026-08-30T13:17:10.514Z"}