{"record":{"id":"dc7d47013d164159","repo":"microsoft/semantic-kernel","slug":"collection-literals-in-filter-expressions-exceed-t","errorCode":null,"errorMessage":"Collection literals in filter expressions exceed the maximum allowed size.","messagePattern":"Collection literals in filter expressions exceed the maximum allowed size\\.","errorType":"exception","errorClass":"VectorStoreOperationException","httpStatus":null,"severity":"error","filePath":"python/semantic_kernel/connectors/in_memory.py","lineNumber":390,"sourceCode":"\n    def _safe_numeric_operation(\n        self,\n        operator_node: ast.AST,\n        left: Any,\n        right: Any,\n        operation: Callable[[float | int, float | int], Any],\n    ) -> Any:\n        \"\"\"Safely evaluate a numeric binary operation.\"\"\"\n        if not isinstance(left, (int, float)) or not isinstance(right, (int, float)):\n            raise VectorStoreOperationException(\n                f\"Operator '{type(operator_node).__name__}' is only allowed for numeric values in filter expressions.\"\n            )\n        return operation(left, right)\n\n    def _ensure_literal_collection_size(self, size: int) -> None:\n        \"\"\"Reject excessively large literal collections.\"\"\"\n        if size > self._max_literal_collection_size:\n            raise VectorStoreOperationException(\n                \"Collection literals in filter expressions exceed the maximum allowed size.\"\n            )\n\n    def _ensure_sequence_result_size(\n        self,\n        left: str | list[Any] | tuple[Any, ...],\n        right: str | list[Any] | tuple[Any, ...],\n        operation: Callable[[Any, Any], Any],\n    ) -> Any:\n        \"\"\"Reject oversized sequence concatenation results.\"\"\"\n        if len(left) + len(right) > self._max_sequence_repeat_size:\n            raise VectorStoreOperationException(\n                \"Sequence operations in filter expressions exceed the maximum allowed size.\"\n            )\n        return operation(left, right)\n\n    def _evaluate_optional(self, node: ast.AST | None, context: Mapping[str, Any]) -> Any:\n        \"\"\"Evaluate an optional AST node.\"\"\"","sourceCodeStart":372,"sourceCodeEnd":408,"githubUrl":"https://github.com/microsoft/semantic-kernel/blob/c028a0c7dc4f0814cdcbaba9d998f187a41197bf/python/semantic_kernel/connectors/in_memory.py#L372-L408","documentation":"Thrown by _ensure_literal_collection_size (in_memory.py:389-391) during evaluation when a list/tuple/set/dict literal has more than max_filter_literal_collection_size elements (default 256). Parse-time also enforces this (in_memory.py:814-825), so the evaluator check is defense-in-depth for the case where limits differ or parsing was bypassed.","triggerScenarios":"A filter containing a literal collection larger than 256 elements, e.g. `lambda x: x.id in [1, 2, ..., 300]`, or a relaxed/custom parse path.","commonSituations":"Pasting a large allowlist/ID set inline into a filter; migrating from a version with different limits.","solutions":["Shrink the inline collection to at most max_filter_literal_collection_size elements.","Pass options.filter as a Python callable instead of a string to bypass string parsing limits where appropriate.","Raise max_filter_literal_collection_size on the collection if the large literal is legitimate.","Split one big filter into a list of smaller filters (options.filter accepts a list)."],"exampleFix":"# before\nVectorSearchOptions(filter=f\"lambda x: x.id in {list(range(300))}\")  # > 256 elements\n# after\nallowed = {i for i in range(300)}\nVectorSearchOptions(filter=lambda x: x.id in allowed)  # callable bypasses parse limits","handlingStrategy":"validation","validationCode":"def collection_within_limit(expr: str, cap: int = 256) -> bool:\n    import ast\n    tree = ast.parse(expr, mode=\"eval\")\n    for n in ast.walk(tree):\n        if isinstance(n, (ast.List, ast.Tuple, ast.Set)) and len(n.elts) > cap:\n            return False\n        if isinstance(n, ast.Dict) and len(n.keys) > cap:\n            return False\n    return True","typeGuard":null,"tryCatchPattern":"try:\n    results = await collection.search(search_type=SearchType.VECTOR, options=opts)\nexcept VectorStoreOperationException as e:\n    logger.warning(\"filter rejected: %s\", e.__cause__ or e)\n    results = None","preventionTips":["Keep inline literal collections small in filters.","Pass large sets as callables or store them as fields."],"tags":["filter","in-memory","resource-limits","semantic-kernel","evaluation"],"backgroundTag":null,"analyzedSha":"c028a0c7dc4f0814cdcbaba9d998f187a41197bf","analyzedAt":"2026-08-13T13:48:05.040Z","schemaVersion":2},"datasetVersion":"2026-08-13T14:17:21.547Z"}