{"record":{"id":"9e7d8ef1903ecfc0","repo":"microsoft/semantic-kernel","slug":"error-evaluating-filter-e","errorCode":null,"errorMessage":"Error evaluating filter: {e}","messagePattern":"Error evaluating filter: (.+?)","errorType":"exception","errorClass":"VectorStoreOperationException","httpStatus":null,"severity":"error","filePath":"python/semantic_kernel/connectors/in_memory.py","lineNumber":735,"sourceCode":"        for idx, key in enumerate(return_records.keys()):\n            if idx >= skip:\n                returned += 1\n                rec = self.inner_storage[key]\n                rec[IN_MEMORY_SCORE_KEY] = return_records[key]\n                yield rec\n                if returned >= top:\n                    break\n\n    def _get_filtered_records(self, options: VectorSearchOptions) -> dict[TKey, AttributeDict]:\n        if not options.filter:\n            return self.inner_storage\n        try:\n            callable_filters = [\n                self._parse_and_validate_filter(filter) if isinstance(filter, str) else filter\n                for filter in ([options.filter] if not isinstance(options.filter, list) else options.filter)\n            ]\n        except Exception as e:\n            raise VectorStoreOperationException(f\"Error evaluating filter: {e}\") from e\n        filtered_records: dict[TKey, AttributeDict] = {}\n        for key, record in self.inner_storage.items():\n            for filter in callable_filters:\n                if self._run_filter(filter, record):\n                    filtered_records[key] = record\n        return filtered_records\n\n    def _parse_and_validate_filter(self, filter_str: str) -> Callable:\n        \"\"\"Parse and validate a string filter as a lambda expression, then return the callable.\n\n        Uses an allowlist approach - only explicitly permitted AST node types and function names\n        are allowed. This can be customized by overriding `allowed_filter_ast_nodes` and\n        `allowed_filter_functions` class attributes.\n        \"\"\"\n        if len(filter_str) > self.max_filter_source_length:\n            raise VectorStoreOperationException(\"Filter string exceeds the maximum allowed length.\")\n\n        try:","sourceCodeStart":717,"sourceCodeEnd":753,"githubUrl":"https://github.com/microsoft/semantic-kernel/blob/c028a0c7dc4f0814cdcbaba9d998f187a41197bf/python/semantic_kernel/connectors/in_memory.py#L717-L753","documentation":"Thrown by _get_filtered_records (in_memory.py:734-735) as VectorStoreOperationException wrapping any exception raised while preparing string filters inside _parse_and_validate_filter. The original error is chained as __cause__, and its text is embedded after 'Error evaluating filter: '. So this is the top-level message users see; the precise reason (oversized, invalid python, disallowed node, blocked attribute, etc.) lives in e.__cause__.","triggerScenarios":"Any string filter that fails parse-time validation, e.g. a typo'd lambda, an oversized string, a disallowed AST node, or a blocked dunder attribute; also fires for a non-string/non-callable filter value that breaks the comprehension.","commonSituations":"Building filter strings dynamically and hitting a limit or a typo; user-supplied filter text that is malformed; passing a dict instead of a string/callable as options.filter.","solutions":["Inspect e.__cause__ (or the text after the colon) to find the specific failure, then apply the matching fix (1334-1339).","If the filter is dynamically built, validate it with ast.parse + a lambda check before passing it to search.","Pass options.filter as a Python callable to bypass string parsing entirely when the filter is trusted and complex.","Catch VectorStoreOperationException at the search boundary and fall back to an unfiltered or simpler query."],"exampleFix":"# before\nopts = VectorSearchOptions(filter=\"x.id == 1\")  # not a lambda -> wrapped error\n# after\nopts = VectorSearchOptions(filter=\"lambda x: x.id == 1\")","handlingStrategy":"try-catch","validationCode":"import ast\n\nSAFE_NODES = {\n    ast.Expression, ast.Lambda, ast.arguments, ast.arg, ast.Compare, ast.BoolOp,\n    ast.UnaryOp, ast.And, ast.Or, ast.Not, ast.Eq, ast.NotEq, ast.Lt, ast.LtE,\n    ast.Gt, ast.GtE, ast.In, ast.NotIn, ast.Is, ast.IsNot, ast.Name, ast.Load,\n    ast.Attribute, ast.Subscript, ast.Slice, ast.Constant, ast.List, ast.Tuple,\n    ast.Set, ast.Dict, ast.BinOp, ast.Add, ast.Sub, ast.Mult, ast.Div, ast.Mod,\n    ast.FloorDiv, ast.Call,\n}\n\ndef preflight_filter(expr: str, *, max_len: int = 2048, max_nodes: int = 128) -> None:\n    if len(expr) > max_len:\n        raise ValueError(\"filter too long\")\n    try:\n        tree = ast.parse(expr, mode=\"eval\")\n    except SyntaxError as e:\n        raise ValueError(f\"invalid python: {e}\") from e\n    if not (isinstance(tree, ast.Expression) and isinstance(tree.body, ast.Lambda)):\n        raise ValueError(\"filter must be a lambda expression\")\n    blocked = {\"__class__\", \"__globals__\", \"__subclasses__\", \"__builtins__\", \"__code__\"}\n    for n in ast.walk(tree):\n        if isinstance(n, ast.Attribute) and n.attr in blocked:\n            raise ValueError(f\"blocked attribute: {n.attr}\")\n        if type(n) not in SAFE_NODES:\n            raise ValueError(f\"disallowed node: {type(n).__name__}\")\n    if sum(1 for _ in ast.walk(tree)) > max_nodes:\n        raise ValueError(\"filter too complex\")\n","typeGuard":null,"tryCatchPattern":"try:\n    results = await collection.search(search_type=SearchType.VECTOR, options=opts)\nexcept VectorStoreOperationException as e:\n    logger.warning(\"filter rejected: %s\", e.__cause__ or e)\n    results = None","preventionTips":["Always read e.__cause__ to find the real filter failure.","Preflight dynamically built filter strings with ast.parse.","Prefer callable filters for trusted, complex logic."],"tags":["filter","in-memory","error-handling","semantic-kernel"],"backgroundTag":null,"analyzedSha":"c028a0c7dc4f0814cdcbaba9d998f187a41197bf","analyzedAt":"2026-08-13T13:48:05.040Z","schemaVersion":2},"datasetVersion":"2026-08-13T14:17:21.547Z"}