{"record":{"id":"88da82611c333cf5","repo":"pandas-dev/pandas","slug":"cannot-process-expression-self-expr-self-88da82","errorCode":null,"errorMessage":"cannot process expression [{self.expr}], [{self}] is not a valid filter","messagePattern":"cannot process expression \\[(.+?)\\], \\[(.+?)\\] is not a valid filter","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"pandas/core/computation/pytables.py","lineNumber":653,"sourceCode":"\n    def __repr__(self) -> str:\n        if self.terms is not None:\n            return pprint_thing(self.terms)\n        return pprint_thing(self.expr)\n\n    def evaluate(self):\n        \"\"\"create and return the numexpr condition and filter\"\"\"\n        try:\n            self.condition = self.terms.prune(ConditionBinOp)\n        except AttributeError as err:\n            raise ValueError(\n                f\"cannot process expression [{self.expr}], [{self}] \"\n                \"is not a valid condition\"\n            ) from err\n        try:\n            self.filter = self.terms.prune(FilterBinOp)\n        except AttributeError as err:\n            raise ValueError(\n                f\"cannot process expression [{self.expr}], [{self}] \"\n                \"is not a valid filter\"\n            ) from err\n\n        return self.condition, self.filter\n\n\nclass TermValue:\n    \"\"\"hold a term value that we use to construct a condition/filter\"\"\"\n\n    def __init__(self, value, converted, kind: str) -> None:\n        assert isinstance(kind, str), kind\n        self.value = value\n        self.converted = converted\n        self.kind = kind\n\n    def tostring(self, encoding) -> str:\n        \"\"\"quote the string if not encoded else encode and return\"\"\"","sourceCodeStart":635,"sourceCodeEnd":671,"githubUrl":"https://github.com/pandas-dev/pandas/blob/71959b8cb9b2459c16e14b34f28b178ccfe14735/pandas/core/computation/pytables.py#L635-L671","documentation":"Raised inside PyTablesExpr.evaluate() (pandas/core/computation/pytables.py:653) when terms.prune(FilterBinOp) throws AttributeError while building the 'filter' half of an HDFStore 'where' clause. PyTables splits every where-expression into a numexpr 'condition' (numeric/temporal comparisons) and a 'filter' (membership on indexers, e.g. columns=['A','B']); if the expression tree cannot be reduced to a FilterBinOp at all, pruning fails and this ValueError is raised. It is distinct from the sibling 'is not a valid condition' message at line 648, which fires when the condition half fails instead.","triggerScenarios":"Calling pd.read_hdf(path, key, where=...) or HDFStore.select(key, where=...) with a where expression whose parsed BinOp tree has no filterable term. Concretely: passing a bare term/constant as where (e.g. store.select('df','df.index[3]')), referencing a column that is not a data_column and not indexable, or combining terms such that pr() in BinOp.prune returns a non-FilterBinOp node whose .filter attribute is missing.","commonSituations":"Migrating a query that worked on a table with data_columns=True to one without them; using a where clause that only ever yields a condition (e.g. 'A > 5') but the code path expects a filter; typos in column names inside the where string; passing a list/PyTablesExpr that collapses to a scalar term; older tutorials showing 'columns=...' filters on tables that were not written with data_columns.","solutions":["Inspect store.select_as_coordinates(key, where=...) output and the table's queryables (e.g. print the Storer/table info) to confirm the column you filter on is indexable / a data_column.","Re-write the HDF5 table with data_columns=True (or index=True for the specific column): df.to_hdf(path, 'df', format='table', data_columns=True) so the column can be used as a filter.","Rewrite the where expression so it produces a valid membership filter (e.g. columns=['A','B']) rather than an expression that only evaluates to a condition; if you only need a condition, expect the 'condition' path, not the 'filter' path.","If arithmetic is involved, note GH#41100: arithmetic inside a where clause is unsupported; precompute a stored column or filter in pandas after reading (df[df['A'] % 3 == 0])."],"exampleFix":"// before\nstore.put('df', df, format='table')\nstore.select('df', \"columns=['A','B']\")  # raises: 'A'/'B' not data_columns\n\n// after\nstore.put('df', df, format='table', data_columns=True)\nstore.select('df', \"columns=['A','B']\")","handlingStrategy":"validation","validationCode":"import pandas as pd\n\n# Before read_hdf/select with a where filter, confirm the column is indexable.\nwith pd.HDFStore(path, mode='r') as store:\n    node = store.get_node(key)\n    # data_columns are the filterable terms\n    table = node.table\n    indexable = set(table.cols._v_colnames) | set(getattr(table.cols, '_v_indexed', []))\n    needed = {'A', 'B'}  # names referenced in your where clause\n    missing = needed - indexable\n    if missing:\n        raise ValueError(f'columns {missing} are not data_columns; re-write with data_columns=True')","typeGuard":"def is_filterable_where(where: str, indexable_cols: set[str]) -> bool:\n    import re, ast\n    # crude: extract identifiers and confirm each is indexable or a literal\n    try:\n        tree = ast.parse(where, mode='eval')\n    except SyntaxError:\n        return False\n    names = {n.id for n in ast.walk(tree) if isinstance(n, ast.Name)}\n    return names <= indexable_cols","tryCatchPattern":"try:\n    result = store.select(key, where=where)\nexcept ValueError as e:\n    if 'is not a valid filter' in str(e):\n        # fall back to full read + in-memory filter\n        df = store.read(key)\n        result = df  # apply filter in pandas here\n    else:\n        raise","preventionTips":["Always write HDF tables with data_columns=True (or index=True for hot columns) so they are filterable.","Validate the where string against the table's known indexable columns before calling select/read_hdf.","Prefer select_as_coordinates for exploratory queries to surface term errors early.","Keep where strings as simple membership/comparison forms documented in PyTablesExpr examples."],"tags":["hdfstore","pytables","query","where-filter"],"analyzedSha":"71959b8cb9b2459c16e14b34f28b178ccfe14735","analyzedAt":"2026-08-07T01:30:20.476Z","schemaVersion":2},"datasetVersion":"2026-08-07T03:17:09.362Z"}