pandas-dev/pandas · error · ValueError

cannot process expression [{self.expr}], [{self}] is not a v

Error message

cannot process expression [{self.expr}], [{self}] is not a valid filter

What it means

Raised inside PyTablesExpr.evaluate() (pandas/core/computation/pytables.py:653) when terms.prune(FilterBinOp) throws AttributeError while building the 'filter' half of an HDFStore 'where' clause. PyTables splits every where-expression into a numexpr 'condition' (numeric/temporal comparisons) and a 'filter' (membership on indexers, e.g. columns=['A','B']); if the expression tree cannot be reduced to a FilterBinOp at all, pruning fails and this ValueError is raised. It is distinct from the sibling 'is not a valid condition' message at line 648, which fires when the condition half fails instead.

Source

Thrown at pandas/core/computation/pytables.py:653

    def __repr__(self) -> str:
        if self.terms is not None:
            return pprint_thing(self.terms)
        return pprint_thing(self.expr)

    def evaluate(self):
        """create and return the numexpr condition and filter"""
        try:
            self.condition = self.terms.prune(ConditionBinOp)
        except AttributeError as err:
            raise ValueError(
                f"cannot process expression [{self.expr}], [{self}] "
                "is not a valid condition"
            ) from err
        try:
            self.filter = self.terms.prune(FilterBinOp)
        except AttributeError as err:
            raise ValueError(
                f"cannot process expression [{self.expr}], [{self}] "
                "is not a valid filter"
            ) from err

        return self.condition, self.filter


class TermValue:
    """hold a term value that we use to construct a condition/filter"""

    def __init__(self, value, converted, kind: str) -> None:
        assert isinstance(kind, str), kind
        self.value = value
        self.converted = converted
        self.kind = kind

    def tostring(self, encoding) -> str:
        """quote the string if not encoded else encode and return"""

View on GitHub (pinned to 71959b8cb9)

Solutions

Inspect store.select_as_coordinates(key, where=...) output and the table's queryables (e.g. print the Storer/table info) to confirm the column you filter on is indexable / a data_column.
Re-write the HDF5 table with data_columns=True (or index=True for the specific column): df.to_hdf(path, 'df', format='table', data_columns=True) so the column can be used as a filter.
Rewrite the where expression so it produces a valid membership filter (e.g. columns=['A','B']) rather than an expression that only evaluates to a condition; if you only need a condition, expect the 'condition' path, not the 'filter' path.
If arithmetic is involved, note GH#41100: arithmetic inside a where clause is unsupported; precompute a stored column or filter in pandas after reading (df[df['A'] % 3 == 0]).

Example fix

// before
store.put('df', df, format='table')
store.select('df', "columns=['A','B']")  # raises: 'A'/'B' not data_columns

// after
store.put('df', df, format='table', data_columns=True)
store.select('df', "columns=['A','B']")

Defensive patterns

Strategy: validation

Validate before calling

import pandas as pd

# Before read_hdf/select with a where filter, confirm the column is indexable.
with pd.HDFStore(path, mode='r') as store:
    node = store.get_node(key)
    # data_columns are the filterable terms
    table = node.table
    indexable = set(table.cols._v_colnames) | set(getattr(table.cols, '_v_indexed', []))
    needed = {'A', 'B'}  # names referenced in your where clause
    missing = needed - indexable
    if missing:
        raise ValueError(f'columns {missing} are not data_columns; re-write with data_columns=True')

Type guard

def is_filterable_where(where: str, indexable_cols: set[str]) -> bool:
    import re, ast
    # crude: extract identifiers and confirm each is indexable or a literal
    try:
        tree = ast.parse(where, mode='eval')
    except SyntaxError:
        return False
    names = {n.id for n in ast.walk(tree) if isinstance(n, ast.Name)}
    return names <= indexable_cols

Try / catch

try:
    result = store.select(key, where=where)
except ValueError as e:
    if 'is not a valid filter' in str(e):
        # fall back to full read + in-memory filter
        df = store.read(key)
        result = df  # apply filter in pandas here
    else:
        raise

Prevention

Always write HDF tables with data_columns=True (or index=True for hot columns) so they are filterable.
Validate the where string against the table's known indexable columns before calling select/read_hdf.
Prefer select_as_coordinates for exploratory queries to surface term errors early.
Keep where strings as simple membership/comparison forms documented in PyTablesExpr examples.

When it happens

Trigger: Calling pd.read_hdf(path, key, where=...) or HDFStore.select(key, where=...) with a where expression whose parsed BinOp tree has no filterable term. Concretely: passing a bare term/constant as where (e.g. store.select('df','df.index[3]')), referencing a column that is not a data_column and not indexable, or combining terms such that pr() in BinOp.prune returns a non-FilterBinOp node whose .filter attribute is missing.

Common situations: Migrating a query that worked on a table with data_columns=True to one without them; using a where clause that only ever yields a condition (e.g. 'A > 5') but the code path expects a filter; typos in column names inside the where string; passing a list/PyTablesExpr that collapses to a scalar term; older tutorials showing 'columns=...' filters on tables that were not written with data_columns.

Related errors

AI-assisted analysis of pandas-dev/pandas@71959b8cb9 (2026-08-07). Data as JSON: /api/errors/88da82611c333cf5. Report an issue: GitHub.