{"record":{"id":"e56036cc556fba5e","repo":"apache/beam","slug":"scalar-expression-s-of-type-s-partitoned-by-non-singleton-s","errorCode":null,"errorMessage":"Scalar expression %s of type %s partitoned by non-singleton %s","messagePattern":"Scalar expression (.+?) of type (.+?) partitoned by non-singleton (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/dataframe/frame_base.py","lineNumber":70,"sourceCode":"  def wrap(cls, expr, split_tuples=True):\n    proxy_type = type(expr.proxy())\n    if proxy_type is tuple and split_tuples:\n\n      def get(ix):\n        return expressions.ComputedExpression(\n            # yapf: disable\n            'get_%d' % ix,\n            lambda t: t[ix],\n            [expr],\n            requires_partition_by=partitionings.Arbitrary(),\n            preserves_partition_by=partitionings.Singleton())\n\n      return tuple(cls.wrap(get(ix)) for ix in range(len(expr.proxy())))\n    elif proxy_type in cls._pandas_type_map:\n      wrapper_type = cls._pandas_type_map[proxy_type]\n    else:\n      if expr.requires_partition_by() != partitionings.Singleton():\n        raise ValueError(\n            'Scalar expression %s of type %s partitoned by non-singleton %s' %\n            (expr, proxy_type, expr.requires_partition_by()))\n      wrapper_type = _DeferredScalar\n    return wrapper_type(expr)\n\n  def _elementwise(\n      self, func, name=None, other_args=(), other_kwargs=None, inplace=False):\n    other_kwargs = other_kwargs or {}\n    return _elementwise_function(\n        func, name, inplace=inplace)(self, *other_args, **other_kwargs)\n\n  def __reduce__(self):\n    return UnusableUnpickledDeferredBase, (str(self), )\n\n\nclass UnusableUnpickledDeferredBase(object):\n  \"\"\"Placeholder object used to break the transitive pickling chain in case a\n  DeferredBase accidentially gets pickled (e.g. as part of globals).","sourceCodeStart":52,"sourceCodeEnd":88,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/dataframe/frame_base.py#L52-L88","documentation":"DeferredFrame.wrap only knows proxy types it has registered in _pandas_type_map; for an unregistered scalar-ish proxy type it falls back to _DeferredScalar, which is only valid for expressions partitioned by Singleton. If the expression's requires_partition_by() is not Singleton, wrapping as a scalar would silently mis-partition data, so a ValueError is raised.","triggerScenarios":"Wrapping an expression whose proxy is a custom/numpy scalar type not in _pandas_type_map while the expression requires non-singleton partitioning — typically from calling frame_base.DeferredFrame.wrap on a user-built expression or a projection producing an unusual type.","commonSituations":"Advanced/library code building custom expressions over dataframes; operations returning proxies of unusual types (e.g. numpy arrays) that need shuffling; hitting this after a pandas upgrade changes an inferred result type.","solutions":["Ensure the expression is elementwise (requires Singleton partitioning) before wrapping it as a scalar.","Reshape the result to a supported pandas type (DataFrame/Series) which wrap handles via its type map.","Use convert.to_pcollection/to_dataframe on the expression instead of wrapping it manually."],"exampleFix":"// before\nwrapper = frame_base.DeferredFrame.wrap(expr)  # expr returns unregistered type\n// after\nresult = convert.to_pcolumn_like(expr)  # or cast proxy to pd.Series first\nproxy = pd.Series(dtype=expr.proxy().dtype)\nwrapper = frame_base.DeferredFrame.wrap(expressions.Bind(expr.func, proxy))","handlingStrategy":"validation","validationCode":"from apache_beam.dataframe.partitionings import Singleton\nif expr.requires_partition_by() != Singleton():\n    raise ValueError('cannot wrap non-singleton expression as scalar')","typeGuard":"def wrappable_as_scalar(expr) -> bool:\n    from apache_beam.dataframe.partitionings import Singleton\n    return (type(expr.proxy()) in frame_base.DeferredFrame._pandas_type_map\n            or expr.requires_partition_by() == Singleton())","tryCatchPattern":"try:\n    frame = DeferredFrame.wrap(expr)\nexcept ValueError as e:\n    if 'non-singleton' in str(e):\n        expr = make_elementwise_equivalent(expr)\n        frame = DeferredFrame.wrap(expr)\n    else:\n        raise","preventionTips":["Only manually wrap expressions whose partitioning requirement is Singleton.","Prefer public convert.to_dataframe/to_pcollection over internal wrap calls.","Cast unusual proxy results (numpy scalars/arrays) into pd.Series/DataFrame before wrapping."],"tags":["python","apache-beam","dataframe-api","partitioning"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}