dbt-labs/dbt-core · error · Exception

{type(df)} is not a supported type for dbt Python materializ

Error message

{type(df)} is not a supported type for dbt Python materialization

What it means

The Fabric Spark (fabricspark) dbt Python table materialization macro validates the object bound to `df` before writing it via df.write.saveAsTable. It only accepts pyspark.pandas DataFrames (converted with to_spark) and pandas DataFrames (converted with spark.createDataFrame); any other type raises this Exception because the adapter cannot materialize it as a Delta table.

Source

Thrown at crates/dbt-loader/src/dbt_macro_assets/dbt-fabricspark/macros/materializations/models/table/table.sql:80

except ImportError:
  pyspark_pandas_api_available = False

# preferentially convert pandas DataFrames to pandas-on-Spark first
# since they know how to convert pandas DataFrames better than `spark.createDataFrame(df)`
# and converting from pandas-on-Spark to Spark DataFrame has no overhead
if pyspark_pandas_api_available and pandas_available and isinstance(df, pandas.core.frame.DataFrame):
  df = pyspark.pandas.frame.DataFrame(df)

# convert to pyspark.sql.dataframe.DataFrame
if isinstance(df, pyspark.sql.dataframe.DataFrame):
  pass  # since it is already a Spark DataFrame
elif pyspark_pandas_api_available and isinstance(df, pyspark.pandas.frame.DataFrame):
  df = df.to_spark()
elif pandas_available and isinstance(df, pandas.core.frame.DataFrame):
  df = spark.createDataFrame(df)
else:
  msg = f"{type(df)} is not a supported type for dbt Python materialization"
  raise Exception(msg)

df.write.mode("overwrite").format("delta").option("overwriteSchema", "true").saveAsTable("{{ target_relation }}")
{%- endmacro -%}

{%macro py_script_comment()%}
# how to execute python model in notebook
# dbt = dbtObj(spark.table)
# df = model(dbt, spark)
{%endmacro%}

View on GitHub (pinned to 0267ce9170)

Solutions

1. Return a supported DataFrame type: pyspark.pandas DataFrame or pandas DataFrame from the model function.
2. Convert a pyspark.sql.DataFrame with `df.to_pandas_on_spark()` (or return a pandas DataFrame from `df.toPandas()`).
3. Verify the function actually returns df rather than implicitly returning None.
4. If using another library's DataFrame (polars, koalas), convert to pandas before returning.

Example fix

# before
def model(dbt, session):
    return spark.read.table('source')  # pyspark.sql.DataFrame

# after
def model(dbt, session):
    df = spark.read.table('source').to_pandas_on_spark()
    return df  # pyspark.pandas.DataFrame

import pyspark.pandas as ps
import pandas as pd
assert isinstance(df, (ps.DataFrame, pd.DataFrame)), f'Unsupported type: {type(df)}'

def is_fabric_supported_df(df) -> bool:
    try:
        import pyspark.pandas
        if isinstance(df, pyspark.pandas.frame.DataFrame):
            return True
    except ImportError:
        pass
    try:
        import pandas
        return isinstance(df, pandas.core.frame.DataFrame)
    except ImportError:
        return False

try:
    write_result = materialize(df)
except Exception as e:
    if 'not a supported type for dbt Python materialization' in str(e):
        logger.error('Fabric Spark models must return pyspark.pandas or pandas DataFrames')
        df = df.to_pandas_on_spark() if hasattr(df, 'to_pandas_on_spark') else None

When it happens

Trigger: A Python model on Fabric Spark whose model function returns None (missing return), or returns an object that is neither a pyspark.pandas DataFrame nor a pandas DataFrame — e.g. a pyspark.sql.DataFrame passed through unusual paths, a list from collect(), a polars or koalas DataFrame — reaching the saveAsTable call.

Common situations: Returning a Spark (pyspark.sql) DataFrame from a notebook where the macro expected pyspark.pandas; forgetting the return statement; returning df.head(), df.collect(), or df.take(n) results; running on a Fabric runtime where the pyspark.pandas availability check fails unexpectedly.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• {type(df)} is not a supported type for dbt Python materializ
• {type(df)} is not a supported type for dbt Python materializ
• {type(df)} is not a supported type for dbt Python materializ
• Schema not found for canonical FQN: {}
• get_table_options: Failed to deserialize config: {e}

AI-assisted analysis of dbt-labs/dbt-core@0267ce9170 (2026-09-07). Data as JSON: /api/errors/d9280004b8da6b5b. Report an issue: GitHub.