{"record":{"id":"3fddefe4f6438637","repo":"pandas-dev/pandas","slug":"unable-to-collapse-joint-filters","errorCode":null,"errorMessage":"unable to collapse Joint Filters","messagePattern":"unable to collapse Joint Filters","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"pandas/core/computation/pytables.py","lineNumber":367,"sourceCode":"                )\n            return None\n\n        if self.is_in_table and len(values) <= self._max_selectors:\n            return None\n        filter_op = self.generate_filter_op()\n        self.filter = (self.lhs.value, filter_op, Index(values))\n        return self\n\n    def generate_filter_op(self, invert: bool = False):\n        if (self.op == \"!=\" and not invert) or (self.op == \"==\" and invert):\n            return lambda axis, vals: ~axis.isin(vals)\n        else:\n            return lambda axis, vals: axis.isin(vals)\n\n\nclass JointFilterBinOp(FilterBinOp):\n    def format(self):\n        raise NotImplementedError(\"unable to collapse Joint Filters\")\n\n    # error: Signature of \"evaluate\" incompatible with supertype \"BinOp\"\n    def evaluate(self) -> Self:  # type: ignore[override]\n        return self\n\n\nclass ConditionBinOp(BinOp):\n    def __repr__(self) -> str:\n        return pprint_thing(f\"[Condition : [{self.condition}]]\")\n\n    def invert(self):\n        \"\"\"invert the condition\"\"\"\n        # if self.condition is not None:\n        #    self.condition = \"~(%s)\" % self.condition\n        # return self\n        raise NotImplementedError(\n            \"cannot use an invert condition when passing to numexpr\"\n        )","sourceCodeStart":349,"sourceCodeEnd":385,"githubUrl":"https://github.com/pandas-dev/pandas/blob/71959b8cb9b2459c16e14b34f28b178ccfe14735/pandas/core/computation/pytables.py#L349-L385","documentation":"Raised by JointFilterBinOp.format in pandas.core.computation.pytables. A JointFilterBinOp is created when two FilterBinOps are combined with a boolean operator; pandas can apply each filter separately but cannot serialize the combination into a single pytables 'filter' representation. Calling .format() on such a joint node raises NotImplementedError. In practice this surfaces when the where clause mixes multiple list-membership filters with boolean connectives that the filter-collapsing logic cannot flatten.","triggerScenarios":"store.select('df', where='col_a == [1,2] & col_b == [3,4]') - two list-membership filters joined by '&' that the engine cannot collapse into one filter expression. Complex compositions of FilterBinOps that hit JointFilterBinOp.format.","commonSituations":"Combining several .isin()-style on-disk filters; nesting AND/OR of equality-list filters; expecting pytables to push down a multi-column list filter.","solutions":["Split into separate selections or apply filters sequentially in pandas: read once, then df[df['a'].isin([1,2]) & df['b'].isin([3,4])].","Reduce to a single filter dimension and apply the other in pandas after select().","If possible, restructure as a single condition (ConditionBinOp) by using scalar equality instead of list membership.","Precompute a combined boolean column and store it as a data_column."],"exampleFix":"# before\nstore.select('df', where='a == [1,2] & b == [3,4]')  # NotImplementedError: unable to collapse Joint Filters\n\n# after (filter in pandas)\ndf = store.get('df')\ndf[df['a'].isin([1, 2]) & df['b'].isin([3, 4])]","handlingStrategy":"fallback","validationCode":"def has_multiple_list_filters(where: str) -> bool:\n    # detects two or more '== [...]' patterns joined by & or |\n    import re\n    list_filters = re.findall(r'==\\s*\\[', where)\n    return len(list_filters) > 1 and ('&' in where or '|' in where)\n\nif has_multiple_list_filters(where):\n    raise NotImplementedError('multiple list filters cannot be collapsed; filter in pandas')","typeGuard":null,"tryCatchPattern":"try:\n    store.select('df', where=where)\nexcept NotImplementedError as e:\n    if 'unable to collapse Joint Filters' in str(e):\n        df = store.get('df')\n        result = df.query(where)\n    else:\n        raise","preventionTips":["Avoid joining multiple list-membership filters in a single where string.","Apply multi-column isin filters in pandas after reading.","Precompute a combined boolean column when frequent filtering is needed."],"tags":["pandas","hdf5","pytables","where","filters"],"analyzedSha":"71959b8cb9b2459c16e14b34f28b178ccfe14735","analyzedAt":"2026-08-07T01:30:20.476Z","schemaVersion":2},"datasetVersion":"2026-08-07T03:17:09.362Z"}