microsoft/semantic-kernel · error · VectorStoreOperationException

Error running filter: {e}

Error message

Error running filter: {e}

What it means

Thrown by InMemoryCollection._run_filter (in_memory.py:861), wrapping any exception raised while executing a validated filter callable against a record. The original exception is chained (from e). This indicates the filter passed validation but failed at evaluation time, almost always due to record shape mismatch.

Source

Thrown at python/semantic_kernel/connectors/in_memory.py:861

        def filter_callable(*args: Any) -> Any:
            if len(args) != len(lambda_param_order):
                raise VectorStoreOperationException(
                    f"Filter expected {len(lambda_param_order)} argument(s), but received {len(args)}."
                )
            context = {
                name: ReadOnlyAttributeDict._wrap_value(value)
                for name, value in zip(lambda_param_order, args, strict=True)
            }
            return evaluator.evaluate(lambda_node.body, context)

        return filter_callable

    def _run_filter(self, filter: Callable, record: AttributeDict[TAKey, TAValue]) -> bool:
        """Run the filter on the record, supporting attribute access."""
        try:
            return filter(ReadOnlyAttributeDict(record))
        except Exception as e:
            raise VectorStoreOperationException(f"Error running filter: {e}") from e

    @override
    def _lambda_parser(self, node: ast.AST) -> Any:
        """Not used by InMemoryCollection, but required by the interface."""
        pass

    def _calculate_vector_similarity(
        self,
        search_vector: Sequence[float | int],
        record_vector: Sequence[float | int],
        distance_func: Callable,
        invert_score: bool = False,
    ) -> float:
        calc = distance_func(record_vector, search_vector)
        if invert_score:
            return 1.0 - float(calc)
        return float(calc)

View on GitHub (pinned to c028a0c7dc)

Solutions

1. Guard attribute access in the filter: "lambda x: getattr(x, 'age', None) is not None and x.age > 18" (getattr is not in the allowlist, so use a callable filter or ensure the field exists).
2. Normalize records so the filtered field always exists with a consistent type before search.
3. Catch VectorStoreOperationException and inspect the chained __cause__ to see the underlying AttributeError/TypeError.
4. Validate the record schema against the filter expression in a dry-run before issuing the search.

Example fix

# before
filter = "lambda x: x.age > 18"  # fails on records missing 'age'

# after (callable with safe access)
filter = lambda r: r.get('age') is not None and r['age'] > 18

# dry-run the filter against a sample record before searching
def filter_dry_runs(filter_str: str, sample_record: dict) -> bool:
    from semantic_kernel.connectors.in_memory import InMemoryCollection
    # parse using the same validator, then evaluate on the sample
    coll = InMemoryCollection(...)
    fn = coll._parse_and_validate_filter(filter_str)
    try:
        return coll._run_filter(fn, sample_record)
    except Exception:
        return False

def record_has_fields(record: dict, fields: list[str]) -> bool:
    return all(f in record and record[f] is not None for f in fields)

from semantic_kernel.exceptions.vector_store_exceptions import VectorStoreOperationException

try:
    results = await collection.search(options)
except VectorStoreOperationException as ex:
    cause = ex.__cause__
    # cause is usually AttributeError/KeyError/TypeError from record-shape mismatch
    if isinstance(cause, (KeyError, AttributeError)):
        options = VectorSearchOptions(filter=lambda x: x.get('age') is not None and x['age'] > 18)

When it happens

Trigger: The filter accesses a field that does not exist on a record (AttributeError/KeyError), applies an operator to incompatible types (TypeError), or calls an allowed function with bad arguments. Example: "lambda x: x.age > 18" evaluated against a record with no 'age' key.

Common situations: Heterogeneous records where some lack the filtered field; schema drift between filter and data; numeric vs string comparison; None values in a comparison; renamed fields.

Related errors

• Use of name '{node.id}' is not allowed in filter expressions
• Call target node type '{type(node.func).__name__}' is not al
• Function '{func_name}' is not allowed in filter expressions.
• Filter expected {len(lambda_param_order)} argument(s), but r
• BERT summary evaluation score ({f1}) is lower than threshold

AI-assisted analysis of microsoft/semantic-kernel@c028a0c7dc (2026-08-13). Data as JSON: /api/errors/6373441ac6ee2c8d. Report an issue: GitHub.