pandas-dev/pandas · error · NotImplementedError

unable to collapse Joint Filters

Error message

unable to collapse Joint Filters

What it means

Raised by JointFilterBinOp.format in pandas.core.computation.pytables. A JointFilterBinOp is created when two FilterBinOps are combined with a boolean operator; pandas can apply each filter separately but cannot serialize the combination into a single pytables 'filter' representation. Calling .format() on such a joint node raises NotImplementedError. In practice this surfaces when the where clause mixes multiple list-membership filters with boolean connectives that the filter-collapsing logic cannot flatten.

Source

Thrown at pandas/core/computation/pytables.py:367

                )
            return None

        if self.is_in_table and len(values) <= self._max_selectors:
            return None
        filter_op = self.generate_filter_op()
        self.filter = (self.lhs.value, filter_op, Index(values))
        return self

    def generate_filter_op(self, invert: bool = False):
        if (self.op == "!=" and not invert) or (self.op == "==" and invert):
            return lambda axis, vals: ~axis.isin(vals)
        else:
            return lambda axis, vals: axis.isin(vals)


class JointFilterBinOp(FilterBinOp):
    def format(self):
        raise NotImplementedError("unable to collapse Joint Filters")

    # error: Signature of "evaluate" incompatible with supertype "BinOp"
    def evaluate(self) -> Self:  # type: ignore[override]
        return self


class ConditionBinOp(BinOp):
    def __repr__(self) -> str:
        return pprint_thing(f"[Condition : [{self.condition}]]")

    def invert(self):
        """invert the condition"""
        # if self.condition is not None:
        #    self.condition = "~(%s)" % self.condition
        # return self
        raise NotImplementedError(
            "cannot use an invert condition when passing to numexpr"
        )

View on GitHub (pinned to 71959b8cb9)

Solutions

1. Split into separate selections or apply filters sequentially in pandas: read once, then df[df['a'].isin([1,2]) & df['b'].isin([3,4])].
2. Reduce to a single filter dimension and apply the other in pandas after select().
3. If possible, restructure as a single condition (ConditionBinOp) by using scalar equality instead of list membership.
4. Precompute a combined boolean column and store it as a data_column.

Example fix

# before
store.select('df', where='a == [1,2] & b == [3,4]')  # NotImplementedError: unable to collapse Joint Filters

# after (filter in pandas)
df = store.get('df')
df[df['a'].isin([1, 2]) & df['b'].isin([3, 4])]

def has_multiple_list_filters(where: str) -> bool:
    # detects two or more '== [...]' patterns joined by & or |
    import re
    list_filters = re.findall(r'==\s*\[', where)
    return len(list_filters) > 1 and ('&' in where or '|' in where)

if has_multiple_list_filters(where):
    raise NotImplementedError('multiple list filters cannot be collapsed; filter in pandas')

try:
    store.select('df', where=where)
except NotImplementedError as e:
    if 'unable to collapse Joint Filters' in str(e):
        df = store.get('df')
        result = df.query(where)
    else:
        raise

When it happens

Trigger: store.select('df', where='col_a == [1,2] & col_b == [3,4]') - two list-membership filters joined by '&' that the engine cannot collapse into one filter expression. Complex compositions of FilterBinOps that hit JointFilterBinOp.format.

Common situations: Combining several .isin()-style on-disk filters; nesting AND/OR of equality-list filters; expecting pytables to push down a multi-column list filter.

Related errors

• name {self.name!r} is not defined
• arithmetic operations are not supported inside an HDFStore '
• Cannot compare {conv_val} of type {type(conv_val)} to {kind}
• query term is not valid [{self}]
• passing a filterable condition to a non-table indexer [{self

AI-assisted analysis of pandas-dev/pandas@71959b8cb9 (2026-08-07). Data as JSON: /api/errors/3fddefe4f6438637. Report an issue: GitHub.