{"record":{"id":"d8970e0d2d670edf","repo":"pola-rs/polars","slug":"cannot-create-string-column-without-an-offsets-buf","errorCode":null,"errorMessage":"cannot create String column without an offsets buffer","messagePattern":"cannot create String column without an offsets buffer","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/interchange/from_dataframe.py","lineNumber":119,"sourceCode":"        buffers[\"validity\"], column, dtype, data_buffer, offset, allow_copy=allow_copy\n    )\n    return pl.Series._from_buffers(dtype, data=data_buffer, validity=validity_buffer)\n\n\ndef _string_column_to_series(column: Column, *, allow_copy: bool) -> Series:\n    if column.size() == 0:\n        return pl.Series(dtype=String)\n    elif not allow_copy:\n        msg = \"string buffers must be converted\"\n        raise CopyNotAllowedError(msg)\n\n    buffers = column.get_buffers()\n    offset = column.offset\n\n    offsets_buffer_info = buffers[\"offsets\"]\n    if offsets_buffer_info is None:\n        msg = \"cannot create String column without an offsets buffer\"\n        raise RuntimeError(msg)\n    offsets_buffer = _construct_offsets_buffer(\n        *offsets_buffer_info, offset, allow_copy=allow_copy\n    )\n\n    buffer, dtype = buffers[\"data\"]\n    data_buffer = _construct_data_buffer(\n        buffer, dtype, buffer.bufsize, offset=0, allow_copy=allow_copy\n    )\n\n    # First construct a Series without a validity buffer\n    # to allow constructing the validity buffer from a sentinel value\n    data_buffers = [data_buffer, offsets_buffer]\n    data = pl.Series._from_buffers(String, data=data_buffers, validity=None)\n\n    # Add the validity buffer if present\n    validity_buffer = _construct_validity_buffer(\n        buffers[\"validity\"], column, String, data, offset, allow_copy=allow_copy\n    )","sourceCodeStart":101,"sourceCodeEnd":137,"githubUrl":"https://github.com/pola-rs/polars/blob/9b5d73fd00236295624374b075d16b1fe6ec6df9/py-polars/src/polars/interchange/from_dataframe.py#L101-L137","documentation":"When ingesting a string column, Polars requires the protocol column's offsets buffer (which delimits variable-length string values). If column.get_buffers()['offsets'] is None, it raises RuntimeError because string boundaries cannot be reconstructed from the data buffer alone. This indicates a broken or incomplete __dataframe__ implementation on the producer side.","triggerScenarios":"pl.from_dataframe(obj) where obj is a custom/hand-rolled or buggy third-party interchange object whose string columns return no offsets buffer in get_buffers(); never happens with Polars' own producer, which always supplies offsets.","commonSituations":"Writing your own interchange adapter for an in-house storage format and forgetting the offsets entry; using an early/unmaintained interop library that returns only data+validity buffers; protocol partial implementations returning None placeholders.","solutions":["Fix the producer: get_buffers() must return {'data': (buffer, dtype), 'validity': ..., 'offsets': (offsets_buffer, (DtypeKind.INT, 64, 'l', '='))} for string columns.","Bypass the producer: convert via a mature path such as pyarrow (obj -> pa.Table -> pl.from_arrow).","Report the bug to the producer library if it is third-party."],"exampleFix":"# producer side, before: \"offsets\": None -> RuntimeError\n# after:\ndef get_buffers(self):\n    return {\n        \"data\": (self._chars_buf, self._chars_dtype),\n        \"validity\": self._validity(),\n        \"offsets\": (self._offsets_buf, (DtypeKind.INT, 64, \"l\", \"=\")),\n    }","handlingStrategy":"validation","validationCode":"def has_offsets(col) -> bool:\n    if col.dtype[0] != DtypeKind.STRING:\n        return True  # offsets only required for strings\n    return col.get_buffers()[\"offsets\"] is not None\n\nbad = [i for i in range(icdf.num_columns()) if not has_offsets(icdf.get_column(i))]\nif bad:\n    raise ValueError(f\"producer bug: string columns {bad} lack an offsets buffer\")","typeGuard":null,"tryCatchPattern":"try:\n    df = pl.from_dataframe(obj)\nexcept RuntimeError as e:\n    if \"without an offsets buffer\" not in str(e):\n        raise\n    # producer bug: route through pyarrow instead, or fix the producer's get_buffers()","preventionTips":["When implementing __dataframe__, always return (buffer, dtype) for 'offsets' on string columns.","Smoke-test custom interchange producers against pl.from_dataframe before deploying them.","Prefer mature producers (pyarrow) over hand-rolled adapters for string data."],"tags":["polars","interchange-protocol","string","offsets-buffer","runtimeerror"],"backgroundTag":"missing-offsets-buffer","analyzedSha":"9b5d73fd00236295624374b075d16b1fe6ec6df9","analyzedAt":"2026-08-19T12:15:06.350Z","contentChangedAt":"2026-08-19T12:15:06.350Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}