{"record":{"id":"b7e2dffbac6eb3e8","repo":"run-llama/llama_index","slug":"query-id-query-id-not-in-responses","errorCode":null,"errorMessage":"Query id {query_id} not in responses","messagePattern":"Query id (.+?) not in responses","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"llama-index-core/llama_index/core/evaluation/dataset_generation.py","lineNumber":88,"sourceCode":"\n    @classmethod\n    def from_qr_pairs(\n        cls,\n        qr_pairs: List[Tuple[str, str]],\n    ) -> QueryResponseDataset:\n        \"\"\"Create from qr pairs.\"\"\"\n        # define ids as simple integers\n        queries = {str(idx): query for idx, (query, _) in enumerate(qr_pairs)}\n        responses = {str(idx): response for idx, (_, response) in enumerate(qr_pairs)}\n        return cls(queries=queries, responses=responses)\n\n    @property\n    def qr_pairs(self) -> List[Tuple[str, str]]:\n        \"\"\"Get pairs.\"\"\"\n        # if query_id not in response, throw error\n        for query_id in self.queries:\n            if query_id not in self.responses:\n                raise ValueError(f\"Query id {query_id} not in responses\")\n\n        return [\n            (self.queries[query_id], self.responses[query_id])\n            for query_id in self.queries\n        ]\n\n    @property\n    def questions(self) -> List[str]:\n        \"\"\"Get questions.\"\"\"\n        return list(self.queries.values())\n\n    def save_json(self, path: str) -> None:\n        \"\"\"Save json.\"\"\"\n        with open(path, \"w\", encoding=\"utf-8\") as f:\n            json.dump(self.model_dump(), f, indent=4)\n\n    @classmethod\n    def from_json(cls, path: str) -> QueryResponseDataset:","sourceCodeStart":70,"sourceCodeEnd":106,"githubUrl":"https://github.com/run-llama/llama_index/blob/afd0fef371831f9bda13e5af7167cf4e981278ab/llama-index-core/llama_index/core/evaluation/dataset_generation.py#L70-L106","documentation":"Raised by the qr_pairs property of the QueryResponseDataset-style class when a query id present in self.queries has no matching key in self.responses. The property iterates queries and requires a one-to-one id mapping before zipping them into (query, response) tuples.","triggerScenarios":"Constructing the object with queries and responses dicts whose keys diverge — e.g. queries={\"0\": ..., \"1\": ...} but responses={\"0\": ...}, or ids generated by different schemes (integers vs strings, or separately enumerated). Then accessing .qr_pairs.","commonSituations":"Building the dataset manually from two separately collected dicts (questions from one file, answers from another with dropped items); mutating queries after construction (adding a query without its response); type mismatches like int keys vs the str(idx) keys produced by from_qr_pairs.","solutions":["Reconstruct the dataset from aligned pairs via QueryResponseDataset.from_qr_pairs(zip(queries, responses)) so ids are generated consistently.","Normalize keys to strings and drop unmatched ids before construction: ids = queries.keys() & responses.keys().","If you add a query later, add its response under the same id at the same time."],"exampleFix":"# before\nds = cls(queries={\"0\": q0, \"1\": q1}, responses={\"0\": r0})\npairs = ds.qr_pairs  # raises: id \"1\" missing from responses\n\n# after\ncommon = set(queries) & set(responses)\nds = cls(\n    queries={k: queries[k] for k in common},\n    responses={k: responses[k] for k in common},\n)\npairs = ds.qr_pairs","handlingStrategy":"validation","validationCode":"missing = set(ds.queries) - set(ds.responses)\nif missing:\n    raise ValueError(f\"responses missing for ids: {sorted(missing)}\")","typeGuard":"def ids_aligned(ds) -> bool:\n    return set(ds.queries.keys()) <= set(ds.responses.keys())","tryCatchPattern":null,"preventionTips":["Prefer from_qr_pairs over manual dict construction so ids are generated consistently.","Keep all keys as strings and mutate queries/responses in tandem."],"tags":["dataset","key-mismatch","evaluation"],"backgroundTag":null,"analyzedSha":"afd0fef371831f9bda13e5af7167cf4e981278ab","analyzedAt":"2026-08-15T05:42:58.429Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}