pandas-dev/pandas · error · ValueError

cannot process expression

Error message

cannot process expression [{self.expr}], [{self}] is not a valid filter

What it means

Raised by PyTablesExpr.evaluate when self.terms.prune(FilterBinOp) raises AttributeError, meaning the expression cannot be reduced to a filter (the membership/in-table lookup path used for == and != against column values). Filters apply after reading via isin-style logic; an expression with no filterable equality/membership term yields this ValueError.

Solutions

Confirm the column exists in the table and is filterable: inspect store.get_storer('df').data_columns and table.colnames.
Use equality for filter-style lookups: store.select('df', where="A == 'x'").
If only a condition (not a filter) is needed, ensure the clause reduces to a numexpr condition (see error 562) so the filter path returning None is acceptable.
Read the frame and filter in pandas when the HDF5 schema doesn't support the filter.

Example fix

// before
store.select('df', where="A == 'x'")  # A not in table -> invalid filter

// after
store.put('df', df, format='table', data_columns=['A'])
store.select('df', where="A == 'x'")

Defensive patterns

Strategy: validation

Validate before calling

storer = store.get_storer('df')
cols = set(getattr(storer, 'data_columns', []) or []) | set(getattr(storer, 'table', None).colnames if hasattr(storer, 'table') else [])
if 'A' not in cols:
    raise ValueError(f"'A' not in table columns {sorted(cols)}; cannot filter")

Try / catch

try:
    selected = store.select('df', where="A == 'x'")
except ValueError as err:
    if 'is not a valid filter' in str(err):
        df = store.get('df')
        selected = df[df['A'] == 'x']
    else:
        raise

Prevention

Confirm filter columns exist in the table before selecting.
Use == / != for filter-style lookups; other ops go through the condition path.
Re-put the table with data_columns when the filterable set changes.

When it happens

Trigger: store.select('df', where='A > 5') where the term tree has no FilterBinOp (a pure condition) AND the prune for FilterBinOp hits a Term lacking a .filter attribute; referencing a column as filterable that is not in the table; combining terms in a way that leaves a non-filter leaf.

Common situations: Querying a column that is not stored in the table (is_in_table is False) with a non-equality op; expecting an in/membership filter on a column that wasn't written; mismatched expression shape after collapsing joint conditions.

Related errors

AI-assisted analysis of pandas-dev/pandas@3b7651241d (2026-08-11). Data as JSON: /api/errors/88da82611c333cf5. Report an issue: GitHub.

Appendix: source

Thrown at pandas/core/computation/pytables.py:653

    def __repr__(self) -> str:
        if self.terms is not None:
            return pprint_thing(self.terms)
        return pprint_thing(self.expr)

    def evaluate(self):
        """create and return the numexpr condition and filter"""
        try:
            self.condition = self.terms.prune(ConditionBinOp)
        except AttributeError as err:
            raise ValueError(
                f"cannot process expression [{self.expr}], [{self}] "
                "is not a valid condition"
            ) from err
        try:
            self.filter = self.terms.prune(FilterBinOp)
        except AttributeError as err:
            raise ValueError(
                f"cannot process expression [{self.expr}], [{self}] "
                "is not a valid filter"
            ) from err

        return self.condition, self.filter


class TermValue:
    """hold a term value that we use to construct a condition/filter"""

    def __init__(self, value, converted, kind: str) -> None:
        assert isinstance(kind, str), kind
        self.value = value
        self.converted = converted
        self.kind = kind

    def tostring(self, encoding) -> str:
        """quote the string if not encoded else encode and return"""

View on GitHub (pinned to 3b7651241d)