agentscope-ai/agentscope · error · ValueError

top_k cannot exceed Elasticsearch's 10000 limit

Error message

top_k cannot exceed Elasticsearch's 10000 limit

What it means

The Elasticsearch kNN search endpoint caps top_k at 10000; passing a larger value raises ValueError before the query is issued. top_k <= 0 simply returns [].

Source

Thrown at src/agentscope/rag/_vdb/_elasticsearch.py:182

        await self.get_client().delete_by_query(
            index=collection,
            query={"term": {"document_id": document_id}},
            conflicts="proceed",
            refresh=self._refresh is not False,
        )

    async def search(
        self,
        collection: str,
        query_vector: list[float],
        top_k: int = 5,
        metadata_filter: dict[str, Any] | None = None,
    ) -> list[VectorSearchResult]:
        """Run an approximate cosine kNN search."""
        if top_k <= 0:
            return []
        if top_k > 10_000:
            raise ValueError("top_k cannot exceed Elasticsearch's 10000 limit")
        num_candidates = min(
            max(self._num_candidates, top_k),
            10_000,
        )
        knn: dict[str, Any] = {
            "field": "vector",
            "query_vector": query_vector,
            "k": top_k,
            "num_candidates": num_candidates,
        }
        filters = self._metadata_filters(metadata_filter)
        if filters:
            knn["filter"] = filters

        response = await self.get_client().search(
            index=collection,
            size=top_k,
            knn=knn,

View on GitHub (pinned to e90f1c7592)

Solutions

1. Cap top_k at 10000
2. Paginate results if you need more than 10000 hits
3. Validate user-supplied top_k before calling search

Example fix

# before
results = await vdb.search(coll, vec, top_k=50_000)
# after
results = await vdb.search(coll, vec, top_k=min(50_000, 10_000))

top_k = max(1, min(top_k, 10_000))

def valid_top_k(k: int) -> bool:
    return isinstance(k, int) and 1 <= k <= 10_000

When it happens

Trigger: search(collection, query, top_k=20000), or top_k derived from user input / config without bounds.

Common situations: Aggressive 'retrieve everything' settings, or copying num_candidates-style large values into top_k; paginated exports that request whole collections in one call.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• num_candidates must be between 1 and 10000
• Invalid logging level: {level}. Must be one of 'INFO', 'DEBU
• The 'reserve_ratio' of the context config must be smaller th
• The 'context_buffer_ratio' of the injection config must be s
• Input validation failed for tool '{tool_call.name}': {e.mess

AI-assisted analysis of agentscope-ai/agentscope@e90f1c7592 (2026-08-28). Data as JSON: /api/errors/056ec6a1e91de9f3. Report an issue: GitHub.