{"record":{"id":"14b26355643d9796","repo":"apache/beam","slug":"reason-nconsider-using-an-allow-non-parallel-operations","errorCode":null,"errorMessage":"{reason}\\nConsider using an allow_non_parallel_operations block if you're sure you want to do this. See https://s.apache.org/dataframe-non-parallel-operations for more information.","messagePattern":"(.+?)\\\\nConsider using an allow_non_parallel_operations block if you're sure you want to do this\\. See https://s\\.apache\\.org/dataframe-non-parallel-operations for more information\\.","errorType":"exception","errorClass":"NonParallelOperation","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/dataframe/expressions.py","lineNumber":350,"sourceCode":"      name: The name of this expression.\n      func: The function that will be used to compute the value of this\n        expression. Should accept arguments of the types returned when\n        evaluating the `args` expressions.\n      args: The list of expressions that will be used to produce inputs to\n        `func`.\n      proxy: (Optional) a proxy object with same type as the objects that this\n        ComputedExpression will produce at execution time. If not provided, a\n        proxy will be generated using `func` and the proxies of `args`.\n      _id: (Optional) a string to uniquely identify this expression.\n      requires_partition_by: The required (common) partitioning of the args.\n      preserves_partition_by: The level of partitioning preserved.\n    \"\"\"\n    if (not _get_allow_non_parallel() and\n        isinstance(requires_partition_by, partitionings.Singleton)):\n      reason = requires_partition_by.reason or (\n          f\"Encountered non-parallelizable form of {name!r}.\")\n\n      raise NonParallelOperation(\n          f\"{reason}\\n\"\n          \"Consider using an allow_non_parallel_operations block if you're \"\n          \"sure you want to do this. See \"\n          \"https://s.apache.org/dataframe-non-parallel-operations for more \"\n          \"information.\")\n    args = tuple(args)\n    if proxy is None:\n      proxy = func(*(arg.proxy() for arg in args))\n    super().__init__(name, proxy, _id)\n    self._func = func\n    self._args = args\n    self._requires_partition_by = requires_partition_by\n    self._preserves_partition_by = preserves_partition_by\n\n  def placeholders(self):\n    if not hasattr(self, '_placeholders'):\n      self._placeholders = frozenset.union(\n          frozenset(), *[arg.placeholders() for arg in self.args()])","sourceCodeStart":332,"sourceCodeEnd":368,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/dataframe/expressions.py#L332-L368","documentation":"The Beam DataFrame API raises NonParallelOperation when an expression requires partitioning all data onto a single bundle (a Singleton partitioning, e.g. sort, quantile, unique) and no allow_non_parallel_operations block is active. The error includes the reason why the operation is non-parallel plus a pointer to the allow_non_parallel_operations escape hatch.","triggerScenarios":"Invoking a pandas operation implemented with requires_partition_by=partitionings.Singleton() (sort_values, nunique, quantile, rank with method, duplicated keep, etc.) outside an allow_non_parallel_operations context.","commonSituations":"Sorting or computing global statistics on a Beam dataframe and expecting distributed execution; migrating pandas pipelines where cheap local operations become globally blocking in Beam; users unaware that certain pandas semantics fundamentally need all data in one place.","solutions":["Wrap the operation in `with default_backend.allow_non_parallel_operations():` (from apache_beam.dataframe) if you accept the single-bundle cost.","Rephrase the logic to avoid the non-parallel operation (e.g. approximate quantiles, per-key sort instead of global).","Use native Beam transforms (beam.ApproximateQuantiles, GroupByKey+sort) for the specific global computation."],"exampleFix":"// before\nresult = df.sort_values('ts')\n// after\nwith allow_non_parallel_operations():\n  result = df.sort_values('ts')","handlingStrategy":"try-catch","validationCode":"from apache_beam.dataframe import expressions\nfrom apache_beam.dataframe import partitionings\n# before calling a suspect op:\n# if expr.requires_partition_by() == partitionings.Singleton(): prepare allow_non_parallel_operations block","typeGuard":"def needs_non_parallel(expr) -> bool:\n    from apache_beam.dataframe.partitionings import Singleton\n    return expr.requires_partition_by() == Singleton()","tryCatchPattern":"from apache_beam.dataframe.frame_base import NonParallelOperation\ntry:\n    result = df.sort_values('col')\nexcept NonParallelOperation as e:\n    with allow_non_parallel_operations():\n        result = df.sort_values('col')","preventionTips":["Know which pandas ops are Singleton-partitioned (sort, unique, quantile, rank) before using them in Beam.","Budget the cost: allow_non_parallel_operations funnels all data to one worker.","Prefer per-key operations (groupby().sort_values()) over global ones."],"tags":["python","apache-beam","dataframe-api","non-parallel-operation"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}