pandas-dev/pandas · error · NotImplementedError

arithmetic operations are not supported inside an HDFStore '

Error message

arithmetic operations are not supported inside an HDFStore 'where' filter; instead store a precomputed column as a data_column and query that, or read the data and apply the filter in pandas (e.g. df[df['A'] % 3 == 0]).

What it means

Raised by BinOp.prune in pandas.core.computation.pytables whenever an arithmetic operator (one of ARITH_OPS_SYMS: + - * / ** // %) appears in an HDFStore 'where' clause. PyTables' on-disk query grammar only supports comparisons and boolean composition; arithmetic would require materializing the column. Per GH#41100 the team chose to raise NotImplementedError with an actionable pointer rather than expand the query grammar, since PyTables support is in maintenance mode.

Source

Thrown at pandas/core/computation/pytables.py:136

    op: str
    queryables: dict[str, Any]
    condition: str | None

    def __init__(self, op: str, lhs, rhs, queryables: dict[str, Any], encoding) -> None:
        super().__init__(op, lhs, rhs)
        self.queryables = queryables
        self.encoding = encoding
        self.condition = None

    def _disallow_scalar_only_bool_ops(self) -> None:
        pass

    def prune(self, klass):
        if self.op in ARITH_OPS_SYMS:
            # GH#41100: arithmetic in a where-clause is not supported. PyTables
            # support is in maintenance mode, so rather than grow the query
            # grammar we raise with a pointer to a working alternative.
            raise NotImplementedError(
                "arithmetic operations are not supported inside an HDFStore "
                "'where' filter; instead store a precomputed column as a "
                "data_column and query that, or read the data and apply the "
                "filter in pandas (e.g. df[df['A'] % 3 == 0])."
            )

        def pr(left, right):
            """create and return a new specialized BinOp from myself"""
            if left is None:
                return right
            elif right is None:
                return left

            k = klass
            if isinstance(left, ConditionBinOp):
                if isinstance(right, ConditionBinOp):
                    k = JointConditionBinOp
                elif isinstance(left, k):

View on GitHub (pinned to 71959b8cb9)

Solutions

1. Precompute the derived column and store it as a data_column: df['A_mod3'] = df['A'] % 3; store.put('df', df, format='table', data_columns=['A_mod3']); then store.select('df', where='A_mod3 == 0').
2. Read the data first and filter in pandas: df = store.get('df'); df[df['A'] % 3 == 0].
3. If the arithmetic is on the comparison value (not the column), move it out: precompute threshold = 3 and write where='A > @threshold' style via TermValue, or just use a literal.

Example fix

# before
store.select('df', where='A % 3 == 0')  # NotImplementedError

# after (precompute column)
df['A_mod3'] = df['A'] % 3
store.put('df', df, format='table', data_columns=['A_mod3'])
store.select('df', where='A_mod3 == 0')
# after (filter in pandas)
df = store.get('df')
df[df['A'] % 3 == 0]

import re
from pandas.core.computation.ops import ARITH_OPS_SYMS

def assert_no_arithmetic_in_where(where: str) -> str:
    # crude check: any arithmetic op token between identifiers
    if re.search(r'[A-Za-z_0-9\]\)]\s*[+\-*/%]|\*\*|//', where):
        raise NotImplementedError(
            f'arithmetic detected in where={where!r}; precompute the column instead'
        )
    return where

from pandas.core.computation.ops import ARITH_OPS_SYMS

def where_has_arithmetic(where: str) -> bool:
    return any(op in where for op in ARITH_OPS_SYMS if op != '-' or ' - ' in where)

try:
    store.select('df', where=where)
except NotImplementedError as e:
    if 'arithmetic operations are not supported' in str(e):
        # precompute derived column or read+filter
        df = store.get('df')
        result = df.query(where)
    else:
        raise

When it happens

Trigger: store.select('df', where='A % 3 == 0'); store.select('df', where='A + B > 10'); pd.read_hdf(path, where='price * qty > 100'). Any expression that computes a new value before comparing.

Common situations: Migrating SQL-like queries that compute on the fly; wanting modulo/range transforms during selection; pre-aggregation logic encoded in the where clause.

Related errors

• name {self.name!r} is not defined
• Cannot compare {conv_val} of type {type(conv_val)} to {kind}
• query term is not valid [{self}]
• passing a filterable condition to a non-table indexer [{self
• unable to collapse Joint Filters

AI-assisted analysis of pandas-dev/pandas@71959b8cb9 (2026-08-07). Data as JSON: /api/errors/b4214346d172170f. Report an issue: GitHub.