{"record":{"id":"056ec6a1e91de9f3","repo":"agentscope-ai/agentscope","slug":"top-k-cannot-exceed-elasticsearch-s-10000-limit","errorCode":null,"errorMessage":"top_k cannot exceed Elasticsearch's 10000 limit","messagePattern":"top_k cannot exceed Elasticsearch's 10000 limit","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/agentscope/rag/_vdb/_elasticsearch.py","lineNumber":182,"sourceCode":"        await self.get_client().delete_by_query(\n            index=collection,\n            query={\"term\": {\"document_id\": document_id}},\n            conflicts=\"proceed\",\n            refresh=self._refresh is not False,\n        )\n\n    async def search(\n        self,\n        collection: str,\n        query_vector: list[float],\n        top_k: int = 5,\n        metadata_filter: dict[str, Any] | None = None,\n    ) -> list[VectorSearchResult]:\n        \"\"\"Run an approximate cosine kNN search.\"\"\"\n        if top_k <= 0:\n            return []\n        if top_k > 10_000:\n            raise ValueError(\"top_k cannot exceed Elasticsearch's 10000 limit\")\n        num_candidates = min(\n            max(self._num_candidates, top_k),\n            10_000,\n        )\n        knn: dict[str, Any] = {\n            \"field\": \"vector\",\n            \"query_vector\": query_vector,\n            \"k\": top_k,\n            \"num_candidates\": num_candidates,\n        }\n        filters = self._metadata_filters(metadata_filter)\n        if filters:\n            knn[\"filter\"] = filters\n\n        response = await self.get_client().search(\n            index=collection,\n            size=top_k,\n            knn=knn,","sourceCodeStart":164,"sourceCodeEnd":200,"githubUrl":"https://github.com/agentscope-ai/agentscope/blob/e90f1c7592896cc95f6e5ee506194f533378247d/src/agentscope/rag/_vdb/_elasticsearch.py#L164-L200","documentation":"The Elasticsearch kNN search endpoint caps top_k at 10000; passing a larger value raises ValueError before the query is issued. top_k <= 0 simply returns [].","triggerScenarios":"search(collection, query, top_k=20000), or top_k derived from user input / config without bounds.","commonSituations":"Aggressive 'retrieve everything' settings, or copying num_candidates-style large values into top_k; paginated exports that request whole collections in one call.","solutions":["Cap top_k at 10000","Paginate results if you need more than 10000 hits","Validate user-supplied top_k before calling search"],"exampleFix":"# before\nresults = await vdb.search(coll, vec, top_k=50_000)\n# after\nresults = await vdb.search(coll, vec, top_k=min(50_000, 10_000))","handlingStrategy":"validation","validationCode":"top_k = max(1, min(top_k, 10_000))","typeGuard":"def valid_top_k(k: int) -> bool:\n    return isinstance(k, int) and 1 <= k <= 10_000","tryCatchPattern":null,"preventionTips":["Cap user-supplied top_k","Paginate for large exports instead of one giant query"],"tags":["elasticsearch","top-k","validation","knn"],"backgroundTag":"invalid-argument-value","analyzedSha":"e90f1c7592896cc95f6e5ee506194f533378247d","analyzedAt":"2026-08-28T18:24:12.087Z","schemaVersion":2},"datasetVersion":"2026-08-28T21:17:43.275Z"}