microsoft/semantic-kernel · error · VectorStoreOperationException

Filter string exceeds the maximum allowed length.

Error message

Filter string exceeds the maximum allowed length.

What it means

Thrown by _parse_and_validate_filter (in_memory.py:750-751) when the raw filter string is longer than max_filter_source_length (default 2048 characters), before any parsing. It is the first guard against oversized filter input.

Source

Thrown at python/semantic_kernel/connectors/in_memory.py:751

            ]
        except Exception as e:
            raise VectorStoreOperationException(f"Error evaluating filter: {e}") from e
        filtered_records: dict[TKey, AttributeDict] = {}
        for key, record in self.inner_storage.items():
            for filter in callable_filters:
                if self._run_filter(filter, record):
                    filtered_records[key] = record
        return filtered_records

    def _parse_and_validate_filter(self, filter_str: str) -> Callable:
        """Parse and validate a string filter as a lambda expression, then return the callable.

        Uses an allowlist approach - only explicitly permitted AST node types and function names
        are allowed. This can be customized by overriding `allowed_filter_ast_nodes` and
        `allowed_filter_functions` class attributes.
        """
        if len(filter_str) > self.max_filter_source_length:
            raise VectorStoreOperationException("Filter string exceeds the maximum allowed length.")

        try:
            tree = ast.parse(filter_str, mode="eval")
        except SyntaxError as e:
            raise VectorStoreOperationException(f"Filter string is not valid Python: {e}") from e

        # Only allow lambda expressions at the top level
        if not (isinstance(tree, ast.Expression) and isinstance(tree.body, ast.Lambda)):
            raise VectorStoreOperationException(
                "Filter string must be a lambda expression, e.g. 'lambda x: x.key == 1'"
            )

        # Get the lambda parameter name(s) to allow them as valid Name nodes
        lambda_node = tree.body
        lambda_param_names = {arg.arg for arg in lambda_node.args.args}
        lambda_param_order = [arg.arg for arg in lambda_node.args.args]
        # Walk the AST to validate all nodes against the allowlist
        for node_count, node in enumerate(ast.walk(tree), start=1):

View on GitHub (pinned to c028a0c7dc)

Solutions

1. Shorten the filter string (e.g. move large literal sets out of the filter).
2. Pass options.filter as a Python callable instead of a string to bypass the source-length limit for trusted logic.
3. Raise max_filter_source_length on the collection instance if a genuinely long filter is required.
4. Split a single long filter into a list of shorter filters (options.filter accepts a list, OR-semantics).

Example fix

# before
collection.max_filter_source_length = 2048  # default
opts = VectorSearchOptions(filter=f"lambda x: x.id in {str(list(range(5000)))}")  # huge
# after
allowed = set(range(5000))
opts = VectorSearchOptions(filter=lambda x: x.id in allowed)  # callable, no length limit

def within_source_limit(expr: str, cap: int = 2048) -> bool:
    return len(expr) <= cap

try:
    results = await collection.search(search_type=SearchType.VECTOR, options=opts)
except VectorStoreOperationException as e:
    logger.warning("filter rejected: %s", e.__cause__ or e)
    results = None

When it happens

Trigger: Passing a very long lambda string, typically one containing a huge inline literal collection or very long string constants.

Common situations: Inlining a large ID allowlist into the filter; generating filters from templates that balloon in size; lowering the limit on a shared collection.

Related errors

• Sequence repetition in filter expressions exceeds the maximu
• Collection literals in filter expressions exceed the maximum
• Sequence operations in filter expressions exceed the maximum
• Filter expression exceeds the maximum allowed complexity.
• Attribute '{node.func.attr}' is not callable in filter expre

AI-assisted analysis of microsoft/semantic-kernel@c028a0c7dc (2026-08-13). Data as JSON: /api/errors/e696fdd0a9f25583. Report an issue: GitHub.