{"record":{"id":"073df0f4d5ca9593","repo":"apache/beam","slug":"append-ignore-index-true-is-order-sensitive-because-it","errorCode":null,"errorMessage":"append(ignore_index=True) is order sensitive because it requires generating a new index based on the order of the data.","messagePattern":"append\\(ignore_index=True\\) is order sensitive because it requires generating a new index based on the order of the data\\.","errorType":"exception","errorClass":"WontImplementError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/dataframe/frames.py","lineNumber":1385,"sourceCode":"  transpose = frame_base._elementwise_method('transpose', base=pd.Series)\n  shape = property(\n      frame_base.wont_implement_method(\n          pd.Series, 'shape', reason=\"non-deferred-result\"))\n\n  @frame_base.with_docs_from(pd.Series, removed_method=PD_VERSION >= (2, 0))\n  @frame_base.args_to_kwargs(pd.Series, removed_method=PD_VERSION >= (2, 0))\n  @frame_base.populate_defaults(pd.Series, removed_method=PD_VERSION >= (2, 0))\n  def append(self, to_append, ignore_index, verify_integrity, **kwargs):\n    \"\"\"``ignore_index=True`` is not supported, because it requires generating an\n    order-sensitive index.\"\"\"\n    if PD_VERSION >= (2, 0):\n      raise frame_base.WontImplementError('append() was removed in Pandas 2.0.')\n    if not isinstance(to_append, DeferredSeries):\n      raise frame_base.WontImplementError(\n          \"append() only accepts DeferredSeries instances, received \" +\n          str(type(to_append)))\n    if ignore_index:\n      raise frame_base.WontImplementError(\n          \"append(ignore_index=True) is order sensitive because it requires \"\n          \"generating a new index based on the order of the data.\",\n          reason=\"order-sensitive\")\n\n    if verify_integrity:\n      # We can verify the index is non-unique within index partitioned data.\n      requires = partitionings.Index()\n    else:\n      requires = partitionings.Arbitrary()\n\n    return frame_base.DeferredFrame.wrap(\n        expressions.ComputedExpression(\n            'append', lambda s, to_append: s.append(\n                to_append, verify_integrity=verify_integrity, **kwargs),\n            [self._expr, to_append._expr],\n            requires_partition_by=requires,\n            preserves_partition_by=partitionings.Arbitrary()))\n","sourceCodeStart":1367,"sourceCodeEnd":1403,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/dataframe/frames.py#L1367-L1403","documentation":"DeferredSeries.append(ignore_index=True) is rejected because generating a fresh 0..n-1 index requires assigning positions based on the order rows arrive — exactly the order dependence Beam cannot guarantee in distributed execution (reason \"order-sensitive\"). Appending with ignore_index=False keeps existing labels and is allowed.","triggerScenarios":"Calling s.append(to_append, ignore_index=True) on a DeferredSeries (with pandas < 2.0, otherwise the removal error fires first).","commonSituations":"Porting pandas code that resets the index after concatenation; expecting a contiguous RangeIndex in a distributed result.","solutions":["Keep meaningful index labels and call append(to_append, ignore_index=False).","Use pd.concat([s, to_append]) and reset_index() after to_pandas() if a clean index is only needed for output.","Drop the index need entirely (e.g. write values only)."],"exampleFix":"// before\ns.append(other, ignore_index=True)\n// after\ns.append(other, ignore_index=False)  # or: s.to_pandas().append(other, ignore_index=True)","handlingStrategy":"validation","validationCode":"if ignore_index:\n    raise ValueError(\"append(ignore_index=True) is order-sensitive in Beam; keep index labels or concat after to_pandas()\")","typeGuard":"def is_index_safe(ignore_index):\n    return not bool(ignore_index)","tryCatchPattern":"from apache_beam.dataframe import frame_base\ntry:\n    combined = s.append(to_append, ignore_index=True)\nexcept frame_base.WontImplementError:\n    combined = s.to_pandas().append(to_append.to_pandas(), ignore_index=True)","preventionTips":["Never pass ignore_index=True to deferred append; preserve labels instead.","Treat any operation generating a fresh positional index as a Beam red flag.","Reset the index only in the final eager stage after to_pandas()."],"tags":["apache-beam","dataframe","pandas","order-sensitivity","index"],"backgroundTag":"order-sensitive-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}