{"record":{"id":"d1ee7c525b9f5d0e","repo":"apache/beam","slug":"grouping-by-a-concrete-ndarray-is-order-sensitive","errorCode":null,"errorMessage":"Grouping by a concrete ndarray is order sensitive.","messagePattern":"Grouping by a concrete ndarray is order sensitive\\.","errorType":"exception","errorClass":"WontImplementError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/dataframe/frames.py","lineNumber":508,"sourceCode":"      to_group = expressions.ComputedExpression(\n          'set_index',\n          set_index, [self._expr, by._expr],\n          requires_partition_by=partitionings.Index(),\n          preserves_partition_by=partitionings.Singleton())\n\n      orig_nlevels = self._expr.proxy().index.nlevels\n      to_group_with_index = expressions.ComputedExpression(\n          'prependindex',\n          prepend_index, [self._expr, by._expr],\n          requires_partition_by=partitionings.Index(),\n          preserves_partition_by=partitionings.Index(\n              list(range(1, orig_nlevels + 1))))\n\n      grouping_columns = []\n      grouping_indexes = [0]\n\n    elif isinstance(by, np.ndarray):\n      raise frame_base.WontImplementError(\n          \"Grouping by a concrete ndarray is order sensitive.\",\n          reason=\"order-sensitive\")\n\n    elif isinstance(self, DeferredDataFrame):\n      if not isinstance(by, list):\n        by = [by]\n      # Find the columns that we need to move into the index so we can group by\n      # them\n      column_names = self._expr.proxy().columns\n      grouping_columns = list(set(by).intersection(column_names))\n      index_names = self._expr.proxy().index.names\n      for label in by:\n        if label not in index_names and label not in self._expr.proxy().columns:\n          raise KeyError(label)\n      grouping_indexes = list(set(by).intersection(index_names))\n\n      if grouping_indexes:\n        if set(by) == set(index_names):","sourceCodeStart":490,"sourceCodeEnd":526,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/dataframe/frames.py#L490-L526","documentation":"groupby(by=<numpy.ndarray>) is rejected because grouping by a concrete ndarray keys groups by data order, which Beam cannot preserve across distributed workers. The Beam DataFrame API only supports grouping by column labels, list of labels, Series, or Index — not raw arrays.","triggerScenarios":"df.groupby(np.array([...])) or df.groupby(by=some_ndarray) on a DeferredDataFrame; also reachable via callers like aggregate, duplicated and drop_duplicates that pass ndarray keys through to groupby.","commonSituations":"Converting pandas code that groups by an external array of keys; building grouping keys programmatically as arrays; duplicated()/drop_duplicates() calls on arrays.","solutions":["Assign the array as a column first: df['key'] = array (as a deferred column), then df.groupby('key').","Group by an existing column name or list of column names.","Wrap the array in a pd.Series/Index that aligns by index if the data supports it.","Collect to pandas with to_pandas() if the operation must be non-deferred."],"exampleFix":"// before\ngroups = df.groupby(np.array(['a','b','a']))\n// after\ndf = df.assign(key=['a','b','a'])\ngroups = df.groupby('key')","handlingStrategy":"type-guard","validationCode":"if isinstance(by, np.ndarray):\n    raise ValueError('Assign the array as a column and group by that column instead')","typeGuard":"def groupby_is_supported(by) -> bool:\n    return not isinstance(by, np.ndarray)","tryCatchPattern":"try:\n    grouped = df.groupby(by)\nexcept frame_base.WontImplementError:\n    df = df.assign(key=by)\n    grouped = df.groupby('key')","preventionTips":["Group only by column labels, lists of labels, or Series","Convert ndarray keys into DataFrame columns before grouping","Check callers like duplicated()/drop_duplicates() for ndarray keys"],"tags":["apache-beam","dataframe","pandas","groupby","order-sensitive"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}