{"record":{"id":"207286cc90167112","repo":"apache/beam","slug":"astype-copy-copy-r-is-not-supported-because-it-relies-on","errorCode":null,"errorMessage":"astype(copy={copy!r}) is not supported because it relies on memory-sharing semantics that are not compatible with the Beam model.","messagePattern":"astype\\(copy=(.+?)\\) is not supported because it relies on memory-sharing semantics that are not compatible with the Beam model\\.","errorType":"exception","errorClass":"WontImplementError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/dataframe/frames.py","lineNumber":637,"sourceCode":"    semantics.\n\n    ``dtype=\"category`` is not supported because the type of the output column\n    depends on the data. Please use ``pd.CategoricalDtype`` with explicit\n    categories instead.\n    \"\"\"\n    requires = partitionings.Arbitrary()\n\n    if errors == \"ignore\":\n      # We need all data in order to ignore errors and propagate the original\n      # data.\n      requires = partitionings.Singleton(\n          reason=(\n              f\"astype(errors={errors!r}) is currently not parallelizable, \"\n              \"because all data must be collected on one node to determine if \"\n              \"the original data should be propagated instead.\"))\n\n    if not copy:\n      raise frame_base.WontImplementError(\n          f\"astype(copy={copy!r}) is not supported because it relies on \"\n          \"memory-sharing semantics that are not compatible with the Beam \"\n          \"model.\")\n\n    # An instance of CategoricalDtype is actualy considered equal to the string\n    # 'category', so we have to explicitly check if dtype is an instance of\n    # CategoricalDtype, and allow it.\n    # See https://github.com/apache/beam/issues/23276\n    if dtype == 'category' and not isinstance(dtype, pd.CategoricalDtype):\n      raise frame_base.WontImplementError(\n          \"astype(dtype='category') is not supported because the type of the \"\n          \"output column depends on the data. Please use pd.CategoricalDtype \"\n          \"with explicit categories instead.\",\n          reason=\"non-deferred-columns\")\n\n    return frame_base.DeferredFrame.wrap(\n        expressions.ComputedExpression(\n            'astype',","sourceCodeStart":619,"sourceCodeEnd":655,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/dataframe/frames.py#L619-L655","documentation":"astype(copy=False) asks pandas to avoid copying and share memory with the original data. Beam's distributed execution model has no shared-memory semantics between workers, so this promise cannot be honored, and the API raises WontImplementError.","triggerScenarios":"df.astype(dtype, copy=False) on any DeferredDataFrame/DeferredSeries (copy defaulting to False in some pandas versions also triggers it).","commonSituations":"Optimization-minded pandas ports that pass copy=False; code written against older pandas where copy defaulted to False; perf-tuned ETL scripts migrated to Beam.","solutions":["Pass copy=True (or omit copy) when calling astype.","If using pandas >= 1.3 where copy defaults differently, explicitly set copy=True to be safe.","Only take astype(copy=False) after converting to real pandas via to_pandas()."],"exampleFix":"// before\ndf = df.astype('int64', copy=False)\n// after\ndf = df.astype('int64', copy=True)","handlingStrategy":"validation","validationCode":"if copy is False:\n    copy = True  # Beam cannot honor copy=False","typeGuard":"def astype_args_are_safe(dtype, copy=True) -> bool:\n    return bool(copy) and not (dtype == 'category' and not isinstance(dtype, pd.CategoricalDtype))","tryCatchPattern":"try:\n    df = df.astype('int64', copy=False)\nexcept frame_base.WontImplementError:\n    df = df.astype('int64', copy=True)","preventionTips":["Never pass copy=False in Beam DataFrame code","Always set copy=True explicitly given pandas default changes across versions","Review perf-only flags when porting pandas code to distributed runtimes"],"tags":["apache-beam","dataframe","pandas","astype","copy"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}