{"record":{"id":"44ad7ca63e9cf750","repo":"pola-rs/polars","slug":"offsets-buffer-must-be-cast-from-polars-dtype-to","errorCode":null,"errorMessage":"offsets buffer must be cast from {polars_dtype} to Int64","messagePattern":"offsets buffer must be cast from (.+?) to Int64","errorType":"exception","errorClass":"CopyNotAllowedError","httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/interchange/from_dataframe.py","lineNumber":238,"sourceCode":"\ndef _construct_offsets_buffer(\n    buffer: Buffer,\n    dtype: Dtype,\n    offset: int,\n    *,\n    allow_copy: bool,\n) -> Series:\n    polars_dtype = dtype_to_polars_dtype(dtype)\n    length = get_buffer_length_in_elements(buffer.bufsize, dtype) - offset\n\n    buffer_info = (buffer.ptr, offset, length)\n    s = pl.Series._from_buffer(polars_dtype, buffer_info, owner=buffer)\n\n    # Polars only supports Int64 offsets\n    if polars_dtype != Int64:\n        if not allow_copy:\n            msg = f\"offsets buffer must be cast from {polars_dtype} to Int64\"\n            raise CopyNotAllowedError(msg)\n        s = s.cast(Int64)\n\n    return s\n\n\ndef _construct_validity_buffer(\n    validity_buffer_info: tuple[Buffer, Dtype] | None,\n    column: Column,\n    column_dtype: PolarsDataType,\n    data: Series,\n    offset: int = 0,\n    *,\n    allow_copy: bool,\n) -> Series | None:\n    null_type, null_value = column.describe_null\n    if null_type == ColumnNullType.NON_NULLABLE or column.null_count == 0:\n        return None\n","sourceCodeStart":220,"sourceCodeEnd":256,"githubUrl":"https://github.com/pola-rs/polars/blob/9b5d73fd00236295624374b075d16b1fe6ec6df9/py-polars/src/polars/interchange/from_dataframe.py#L220-L256","documentation":"Polars string offsets are Int64. When an imported string column's offsets buffer uses another width (Int32 is the common Arrow/interchange layout, Int16/Int32 from other producers), _construct_offsets_buffer must cast the offsets Series to Int64, which allocates; with allow_copy=False this raises CopyNotAllowedError.","triggerScenarios":"pl.from_dataframe(obj, allow_copy=False) over any string (or string-categorical) column whose offsets buffer dtype is not Int64 — e.g. Arrow string arrays that use int32 offsets by default.","commonSituations":"Zero-copy ingestion of text data from Arrow-based producers; assuming string columns can be imported copy-free; large-string vs string layout differences between producers.","solutions":["Call pl.from_dataframe(obj, allow_copy=True) to permit the Int64 cast.","Have the producer emit Int64 offsets (e.g. Arrow large_utf8-style layout).","Catch CopyNotAllowedError and retry with allow_copy=True."],"exampleFix":"# before\ndf = pl.from_dataframe(obj, allow_copy=False)  # int32 offsets -> CopyNotAllowedError\n\n# after\ndf = pl.from_dataframe(obj, allow_copy=True)   # offsets cast to Int64","handlingStrategy":"fallback","validationCode":"def needs_offsets_cast(icdf) -> bool:\n    for i in range(icdf.num_columns()):\n        col = icdf.get_column(i)\n        if col.dtype[0] != DtypeKind.STRING:\n            continue\n        offsets = col.get_buffers()[\"offsets\"]\n        if offsets is not None and offsets[1][:2] != (DtypeKind.INT, 64):\n            return True\n    return False\n\nallow_copy = not needs_offsets_cast(obj.__dataframe__())","typeGuard":null,"tryCatchPattern":"from polars.interchange.protocol import CopyNotAllowedError\n\ntry:\n    df = pl.from_dataframe(obj, allow_copy=False)\nexcept CopyNotAllowedError:\n    df = pl.from_dataframe(obj, allow_copy=True)  # allows the Int64 offsets cast","preventionTips":["Int32 offsets are the Arrow default — plan for the cast whenever text crosses the interchange boundary.","Use large_string-style Int64 offsets on the producer for copy-sensitive paths."],"tags":["polars","interchange-protocol","string","offsets","allow-copy","copynotallowederror"],"backgroundTag":"copy-not-allowed","analyzedSha":"9b5d73fd00236295624374b075d16b1fe6ec6df9","analyzedAt":"2026-08-19T12:15:06.350Z","contentChangedAt":"2026-08-19T12:15:06.350Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}