{"record":{"id":"a36e30970fc2112b","repo":"pola-rs/polars","slug":"include-file-paths-cannot-be-used-with-use-pyar","errorCode":null,"errorMessage":"`include_file_paths` cannot be used with `use_pyarrow=True`","messagePattern":"`include_file_paths` cannot be used with `use_pyarrow=True`","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/io/parquet/functions.py","lineNumber":235,"sourceCode":"    with `LazyFrame` s.\n\n    \"\"\"\n    if schema is not None:\n        msg = \"the `schema` parameter of `read_parquet` is considered unstable.\"\n        issue_unstable_warning(msg)\n\n    if hive_schema is not None:\n        msg = \"the `hive_schema` parameter of `read_parquet` is considered unstable.\"\n        issue_unstable_warning(msg)\n\n    # Dispatch to pyarrow if requested\n    if use_pyarrow:\n        if n_rows is not None:\n            msg = \"`n_rows` cannot be used with `use_pyarrow=True`\"\n            raise ValueError(msg)\n        if include_file_paths is not None:\n            msg = \"`include_file_paths` cannot be used with `use_pyarrow=True`\"\n            raise ValueError(msg)\n        if schema is not None:\n            msg = \"`schema` cannot be used with `use_pyarrow=True`\"\n            raise ValueError(msg)\n        if hive_schema is not None:\n            msg = (\n                \"cannot use `hive_partitions` with `use_pyarrow=True`\"\n                \"\\n\\nHint: Pass `pyarrow_options` instead with a 'partitioning' entry.\"\n            )\n            raise TypeError(msg)\n        return _read_parquet_with_pyarrow(\n            source,\n            columns=columns,\n            storage_options=storage_options,\n            pyarrow_options=pyarrow_options,\n            memory_map=memory_map,\n            rechunk=rechunk,\n        )\n","sourceCodeStart":217,"sourceCodeEnd":253,"githubUrl":"https://github.com/pola-rs/polars/blob/df599052daf96e7a9cc30a3b0c6bd25d6947e3c0/py-polars/src/polars/io/parquet/functions.py#L217-L253","documentation":"ValueError raised by pl.read_parquet when use_pyarrow=True is combined with include_file_paths. include_file_paths adds a column with each source file's path and is implemented only in polars' native multi-file reader; the pyarrow dispatch path (_read_parquet_with_pyarrow) has no equivalent bridge, so the combination is rejected before reading.","triggerScenarios":"pl.read_parquet(['a.parquet','b.parquet'], use_pyarrow=True, include_file_paths='source'). Any non-None include_file_paths with use_pyarrow=True triggers it.","commonSituations":"Reading a directory/glob of partitioned parquet files with the pyarrow engine while wanting provenance per row; migrating a working native-engine pipeline to use_pyarrow for compatibility and forgetting the path column is native-only.","solutions":["Remove use_pyarrow so the native engine runs — it supports include_file_paths natively.","Keep pyarrow and re-add provenance afterwards by reading files one by one and calling df.with_columns(pl.lit(path).alias(include_file_paths)), concatenating the results.","If pyarrow partitioning metadata is what you need, use the native reader's hive_partitioning support or pyarrow.dataset directly and construct the path column from fragment info."],"exampleFix":"# before\npl.read_parquet('part-*.parquet', use_pyarrow=True, include_file_paths='src')\n\n# after\npl.read_parquet('part-*.parquet', include_file_paths='src')  # native engine","handlingStrategy":"validation","validationCode":"PYARROW_INCOMPATIBLE = {'n_rows', 'include_file_paths', 'schema', 'hive_schema'}\n\ndef read_parquet_safe(path, *, use_pyarrow=False, **kwargs):\n    if use_pyarrow:\n        clash = PYARROW_INCOMPATIBLE & kwargs.keys()\n        if clash:\n            raise ValueError(f'remove {clash} or disable use_pyarrow')\n    return pl.read_parquet(path, use_pyarrow=use_pyarrow, **kwargs)","typeGuard":null,"tryCatchPattern":"try:\n    df = pl.read_parquet(path, use_pyarrow=True, include_file_paths='src')\nexcept ValueError as e:\n    if 'include_file_paths' in str(e):\n        df = pl.read_parquet(path, include_file_paths='src')  # native\n    else:\n        raise","preventionTips":["Treat include_file_paths/schema/n_rows/hive_schema as native-engine features.","When adding file provenance, prefer the native multi-file reader.","Gate use_pyarrow behind an explicit compatibility check in shared code."],"tags":["parquet","pyarrow","parameter-conflict","multi-file","io"],"backgroundTag":null,"analyzedSha":"df599052daf96e7a9cc30a3b0c6bd25d6947e3c0","analyzedAt":"2026-08-16T12:10:03.978Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}