{"record":{"id":"d9280004b8da6b5b","repo":"dbt-labs/dbt-core","slug":"type-df-is-not-a-supported-type-for-dbt-python-d92800","errorCode":null,"errorMessage":"{type(df)} is not a supported type for dbt Python materialization","messagePattern":"(.+?) is not a supported type for dbt Python materialization","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"crates/dbt-loader/src/dbt_macro_assets/dbt-fabricspark/macros/materializations/models/table/table.sql","lineNumber":80,"sourceCode":"except ImportError:\r\n  pyspark_pandas_api_available = False\r\n\r\n# preferentially convert pandas DataFrames to pandas-on-Spark first\r\n# since they know how to convert pandas DataFrames better than `spark.createDataFrame(df)`\r\n# and converting from pandas-on-Spark to Spark DataFrame has no overhead\r\nif pyspark_pandas_api_available and pandas_available and isinstance(df, pandas.core.frame.DataFrame):\r\n  df = pyspark.pandas.frame.DataFrame(df)\r\n\r\n# convert to pyspark.sql.dataframe.DataFrame\r\nif isinstance(df, pyspark.sql.dataframe.DataFrame):\r\n  pass  # since it is already a Spark DataFrame\r\nelif pyspark_pandas_api_available and isinstance(df, pyspark.pandas.frame.DataFrame):\r\n  df = df.to_spark()\r\nelif pandas_available and isinstance(df, pandas.core.frame.DataFrame):\r\n  df = spark.createDataFrame(df)\r\nelse:\r\n  msg = f\"{type(df)} is not a supported type for dbt Python materialization\"\r\n  raise Exception(msg)\r\n\r\ndf.write.mode(\"overwrite\").format(\"delta\").option(\"overwriteSchema\", \"true\").saveAsTable(\"{{ target_relation }}\")\r\n{%- endmacro -%}\r\n\r\n{%macro py_script_comment()%}\r\n# how to execute python model in notebook\r\n# dbt = dbtObj(spark.table)\r\n# df = model(dbt, spark)\r\n{%endmacro%}\r\n","sourceCodeStart":62,"sourceCodeEnd":90,"githubUrl":"https://github.com/dbt-labs/dbt-core/blob/0267ce9170576975b76b64ce856b2e5848e96617/crates/dbt-loader/src/dbt_macro_assets/dbt-fabricspark/macros/materializations/models/table/table.sql#L62-L90","documentation":"The Fabric Spark (fabricspark) dbt Python table materialization macro validates the object bound to `df` before writing it via df.write.saveAsTable. It only accepts pyspark.pandas DataFrames (converted with to_spark) and pandas DataFrames (converted with spark.createDataFrame); any other type raises this Exception because the adapter cannot materialize it as a Delta table.","triggerScenarios":"A Python model on Fabric Spark whose model function returns None (missing return), or returns an object that is neither a pyspark.pandas DataFrame nor a pandas DataFrame — e.g. a pyspark.sql.DataFrame passed through unusual paths, a list from collect(), a polars or koalas DataFrame — reaching the saveAsTable call.","commonSituations":"Returning a Spark (pyspark.sql) DataFrame from a notebook where the macro expected pyspark.pandas; forgetting the return statement; returning df.head(), df.collect(), or df.take(n) results; running on a Fabric runtime where the pyspark.pandas availability check fails unexpectedly.","solutions":["Return a supported DataFrame type: pyspark.pandas DataFrame or pandas DataFrame from the model function.","Convert a pyspark.sql.DataFrame with `df.to_pandas_on_spark()` (or return a pandas DataFrame from `df.toPandas()`).","Verify the function actually returns df rather than implicitly returning None.","If using another library's DataFrame (polars, koalas), convert to pandas before returning."],"exampleFix":"# before\ndef model(dbt, session):\n    return spark.read.table('source')  # pyspark.sql.DataFrame\n\n# after\ndef model(dbt, session):\n    df = spark.read.table('source').to_pandas_on_spark()\n    return df  # pyspark.pandas.DataFrame","handlingStrategy":"type-guard","validationCode":"import pyspark.pandas as ps\nimport pandas as pd\nassert isinstance(df, (ps.DataFrame, pd.DataFrame)), f'Unsupported type: {type(df)}'","typeGuard":"def is_fabric_supported_df(df) -> bool:\n    try:\n        import pyspark.pandas\n        if isinstance(df, pyspark.pandas.frame.DataFrame):\n            return True\n    except ImportError:\n        pass\n    try:\n        import pandas\n        return isinstance(df, pandas.core.frame.DataFrame)\n    except ImportError:\n        return False","tryCatchPattern":"try:\n    write_result = materialize(df)\nexcept Exception as e:\n    if 'not a supported type for dbt Python materialization' in str(e):\n        logger.error('Fabric Spark models must return pyspark.pandas or pandas DataFrames')\n        df = df.to_pandas_on_spark() if hasattr(df, 'to_pandas_on_spark') else None","preventionTips":["On Fabric Spark, return pyspark.pandas DataFrames (or convert with to_pandas_on_spark()).","Convert pyspark.sql.DataFrame via toPandas()/to_pandas_on_spark() before returning.","Verify the return statement exists and returns df itself, not a collected view of it.","Confirm the Fabric runtime exposes the pyspark.pandas API the adapter checks for."],"tags":["python-model","dataframe","fabric-spark","materialization"],"backgroundTag":"type-mismatch","analyzedSha":"0267ce9170576975b76b64ce856b2e5848e96617","analyzedAt":"2026-09-07T21:53:39.732Z","contentChangedAt":"2026-09-07T21:53:39.732Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}