{"record":{"id":"ce6101e1c0cbf38b","repo":"pola-rs/polars","slug":"index-positions-should-be-smaller-than-2-32","errorCode":null,"errorMessage":"index positions should be smaller than 2^32","messagePattern":"index positions should be smaller than 2\\^32","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/_utils/getitem.py","lineNumber":371,"sourceCode":"    idx_type = get_index_type()\n\n    if s.dtype == idx_type:\n        return s\n\n    if not s.dtype.is_integer():\n        if s.dtype == Boolean:\n            _raise_on_boolean_mask()\n        else:\n            msg = f\"cannot treat Series of type {s.dtype} as indices\"\n            raise TypeError(msg)\n\n    if s.len() == 0:\n        return pl.Series(s.name, [], dtype=idx_type)\n\n    if idx_type == UInt32:\n        if s.dtype in {Int64, UInt64} and s.max() >= U32_MAX:  # type: ignore[operator]\n            msg = \"index positions should be smaller than 2^32\"\n            raise ValueError(msg)\n        if s.dtype == Int64 and s.min() < -U32_MAX:  # type: ignore[operator]\n            msg = \"index positions should be greater than or equal to -2^32\"\n            raise ValueError(msg)\n\n    if s.dtype.is_signed_integer():\n        if s.min() < 0:  # type: ignore[operator]\n            if idx_type == UInt32:\n                idxs = s.cast(Int32) if s.dtype in {Int8, Int16} else s\n            else:\n                idxs = s.cast(Int64) if s.dtype in {Int8, Int16, Int32} else s\n\n            # Update negative indexes to absolute indexes.\n            return (\n                idxs.to_frame()\n                .select(\n                    F.when(F.col(idxs.name) < 0)\n                    .then(size + F.col(idxs.name))\n                    .otherwise(F.col(idxs.name))","sourceCodeStart":353,"sourceCodeEnd":389,"githubUrl":"https://github.com/pola-rs/polars/blob/df599052daf96e7a9cc30a3b0c6bd25d6947e3c0/py-polars/src/polars/_utils/getitem.py#L353-L389","documentation":"Default polars wheels build with a 32-bit index dtype (UInt32). _convert_series_to_indices (getitem.py:371) rejects Int64/UInt64 Series whose maximum is >= 2**32 because such positions cannot be represented; the check only exists when get_index_type() == UInt32, so the polars-u64-idx build is unaffected.","triggerScenarios":"df[pl.Series([2**32], dtype=pl.UInt64)]; df[pl.Series([5_000_000_000], dtype=pl.Int64)]; positional gather into frames with more than ~4.29 billion rows on default wheels.","commonSituations":"Very large datasets on default wheels; business IDs (large integers) mistakenly used as row positions instead of values for is_in/join.","solutions":["If the values are IDs, not positions, use df.filter(pl.col('id').is_in(ids)) or join instead of indexing","If positions are genuinely huge, install the 64-bit index wheel: pip install polars-u64-idx (it imports as polars)","Filter/collect a smaller frame first so positions fit in 32 bits"],"exampleFix":"# before\ndf[pl.Series(ids, dtype=pl.UInt64)]  # huge values used as positions\n\n# after\ndf.filter(pl.col(\"id\").is_in(ids))","handlingStrategy":"validation","validationCode":"U32_MAX = 2**32\nif idx.dtype in (pl.Int64, pl.UInt64) and idx.len() and idx.max() >= U32_MAX:\n    raise ValueError(\"positions exceed 32-bit index range; use filter/join or polars-u64-idx\")\ndf[idx]","typeGuard":"def fits_u32(s: \"pl.Series\") -> bool:\n    return not (s.dtype in (pl.Int64, pl.UInt64) and s.len() and (s.max() or 0) >= 2**32)","tryCatchPattern":null,"preventionTips":["Keep row positions below 2**32 on default wheels","Use join/filter on an ID column when values look like large integers","Switch to polars-u64-idx for frames beyond ~4.29 billion rows"],"tags":["python","polars","indices","uint32","limits","valueerror"],"backgroundTag":null,"analyzedSha":"df599052daf96e7a9cc30a3b0c6bd25d6947e3c0","analyzedAt":"2026-08-16T12:10:03.978Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}