{"record":{"id":"68656507e37536e1","repo":"pola-rs/polars","slug":"dataframe-contains-unsupported-data-types-overla","errorCode":null,"errorMessage":"dataframe contains unsupported data types: {overlap!r}","messagePattern":"dataframe contains unsupported data types: (.+?)","errorType":"validation","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/io/delta/_utils.py","lineNumber":107,"sourceCode":"    return dl_tbl\n\n\ndef _check_if_delta_available() -> None:\n    if not _DELTALAKE_AVAILABLE:\n        msg = \"deltalake is not installed\\n\\nPlease run: pip install deltalake\"\n        raise ModuleNotFoundError(msg)\n\n\ndef _check_for_unsupported_types(dtypes: list[DataType]) -> None:\n    schema_dtypes = unpack_dtypes(*dtypes)\n    unsupported_types = {Time, Null}\n    # Note that this overlap check does NOT work correctly for Categorical, so\n    # if Categorical is added back to unsupported_types a different check will\n    # need to be used.\n\n    if overlap := schema_dtypes & unsupported_types:\n        msg = f\"dataframe contains unsupported data types: {overlap!r}\"\n        raise TypeError(msg)\n\n\ndef _extract_table_statistics_from_delta_add_actions(\n    add_actions_df: DataFrame,\n    *,\n    filter_columns: list[str],\n    schema: SchemaDict,\n    verbose: bool,\n) -> DataFrame | None:\n    import polars as pl\n\n    if \"num_records\" not in add_actions_df:\n        if verbose:\n            eprint(\n                \"scan_delta: statistics load failed: 'num_records' column not present\"\n            )\n\n        return None","sourceCodeStart":89,"sourceCodeEnd":125,"githubUrl":"https://github.com/pola-rs/polars/blob/fe841f959ef4d2ceefc05a310d33ed7b1ab24e5e/py-polars/src/polars/io/delta/_utils.py#L89-L125","documentation":"_check_for_unsupported_types rejects DataFrames whose schema contains dtypes the deltalake writer cannot round-trip (currently Time and Null). polars raises TypeError listing the offending dtype overlap.","triggerScenarios":"pl.write_delta(df, ...) or pl.sink_delta(df, ...) where df.schema includes pl.Time or pl.Null columns.","commonSituations":"Writing frames parsed from time-only strings producing pl.Time, or empty/typed-null columns from joins or select of literals.","solutions":["Cast pl.Time columns to pl.String (or Duration) before writing","Cast pl.Null columns to a concrete type (e.g. pl.Int32) or drop them","Update polars/deltalake, as the unsupported set may shrink in newer versions"],"exampleFix":"// before\npl.write_delta(df, uri)  # df has a pl.Time column\n// after\ndf = df.with_columns(pl.col('event_time').cast(pl.String))\npl.write_delta(df, uri)","handlingStrategy":"validation","validationCode":"bad = {pl.Time, pl.Null}\nif set(df.schema.dtypes()) & bad:\n    df = df.with_columns(\n        pl.col(c).cast(pl.String) for c, t in df.schema.items() if t == pl.Time\n    ).drop([c for c, t in df.schema.items() if t == pl.Null])","typeGuard":null,"tryCatchPattern":"try:\n    pl.write_delta(df, uri)\nexcept TypeError as e:\n    if 'unsupported data types' in str(e):\n        df = sanitize_dtypes(df)\n        pl.write_delta(df, uri)","preventionTips":["Cast pl.Time to String/Duration and pl.Null columns to concrete types before writing","Assert expected schema with df.collect_schema() before delta writes","Add a schema check in pipeline entry points"],"tags":["python","polars","delta-lake","dtype","unsupported-dtype"],"backgroundTag":"unsupported-dtype","analyzedSha":"fe841f959ef4d2ceefc05a310d33ed7b1ab24e5e","analyzedAt":"2026-09-18T22:14:11.667Z","contentChangedAt":"2026-09-18T22:14:11.667Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}