{"record":{"id":"5077d619356c5876","repo":"apache/beam","slug":"concat-ignore-index","errorCode":null,"errorMessage":"concat(ignore_index)","messagePattern":"concat\\(ignore_index\\)","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/dataframe/pandas_top_level_functions.py","lineNumber":100,"sourceCode":"  bdate_range = _defer_to_pandas('bdate_range')\n\n  @staticmethod\n  @frame_base.args_to_kwargs(pd)\n  @frame_base.populate_defaults(pd)\n  def concat(\n      objs,\n      axis,\n      join,\n      ignore_index,\n      keys,\n      levels,\n      names,\n      verify_integrity,\n      sort,\n      copy):\n\n    if ignore_index:\n      raise NotImplementedError('concat(ignore_index)')\n    if levels:\n      raise NotImplementedError('concat(levels)')\n\n    if isinstance(objs, Mapping):\n      if keys is None:\n        keys = list(objs.keys())\n      objs = [objs[k] for k in keys]\n    else:\n      objs = list(objs)\n\n    if keys is None:\n      preserves_partitioning = partitionings.Arbitrary()\n    else:\n      # Index 0 will be a new index for keys, only partitioning by the original\n      # indexes (1 to N) will be preserved.\n      nlevels = min(o._expr.proxy().index.nlevels for o in objs)\n      preserves_partitioning = partitionings.Index(\n          [i for i in range(1, nlevels + 1)])","sourceCodeStart":82,"sourceCodeEnd":118,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/dataframe/pandas_top_level_functions.py#L82-L118","documentation":"The Beam DataFrame API's pd.concat wrapper does not support ignore_index=True, because renumbering indexes across concatenated frames conflicts with its distributed index-preserving semantics. Passing ignore_index=True raises NotImplementedError early.","triggerScenarios":"Calling pd.concat(..., ignore_index=True) on deferred Beam dataframes within a Beam DataFrame transform.","commonSituations":"Porting pandas code that resets the index on concat; combining frames with duplicate index values and expecting a fresh default index.","solutions":["Use ignore_index=False (the default) and call reset_index() on the result if a fresh index is needed","Manually drop/rebuild the index after concat: df.reset_index(drop=True) on the deferred result","Restructure the pipeline so frames are concatenated outside Beam (in regular pandas) if index reset semantics are essential"],"exampleFix":"// before\npd.concat([df1, df2], ignore_index=True)\n// after\npd.concat([df1, df2]).reset_index(drop=True)","handlingStrategy":"fallback","validationCode":"if ignore_index:\n    ignore_index = False  # not supported; reset index after concat instead","typeGuard":null,"tryCatchPattern":"try:\n    out = pd.concat(objs, ignore_index=ignore_index)\nexcept NotImplementedError:\n    out = pd.concat(objs).reset_index(drop=True)","preventionTips":["Replace ignore_index=True with a post-concat reset_index() in Beam dataframe code","Audit pandas-to-Beam ports for concat options","Prefer index-preserving semantics; treat index reset as an explicit later step"],"tags":["python","apache-beam","dataframe","pandas","not-implemented"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}