dbt-labs/dbt-core · error · Exception

{type(df)} is not a supported type for dbt Python materializ

Error message

{type(df)} is not a supported type for dbt Python materialization

What it means

Raised by the dbt-bigquery Python model materialization macro when the Python model's returned DataFrame is neither a supported Spark, Koalas, nor pandas DataFrame. The macro inspects the df returned by the model's Python code and only knows how to convert pandas (via spark.createDataFrame) and Koalas (via to_spark); any other type (or a df that failed to import/convert) hits this exception.

Source

Thrown at crates/dbt-loader/src/dbt_macro_assets/dbt-bigquery/macros/materializations/table.sql:114

# since they know how to convert pandas DataFrames better than `spark.createDataFrame(df)`
# and converting from pandas-on-Spark to Spark DataFrame has no overhead
if pyspark_pandas_api_available and pandas_available and isinstance(df, pandas.core.frame.DataFrame):
  df = pyspark.pandas.frame.DataFrame(df)
elif koalas_available and pandas_available and isinstance(df, pandas.core.frame.DataFrame):
  df = databricks.koalas.frame.DataFrame(df)

# convert to pyspark.sql.dataframe.DataFrame
if isinstance(df, pyspark.sql.dataframe.DataFrame):
  pass  # since it is already a Spark DataFrame
elif pyspark_pandas_api_available and isinstance(df, pyspark.pandas.frame.DataFrame):
  df = df.to_spark()
elif koalas_available and isinstance(df, databricks.koalas.frame.DataFrame):
  df = df.to_spark()
elif pandas_available and isinstance(df, pandas.core.frame.DataFrame):
  df = spark.createDataFrame(df)
else:
  msg = f"{type(df)} is not a supported type for dbt Python materialization"
  raise Exception(msg)

# For writeMethod we need to use "indirect" if materializing a partitioned table
# otherwise we can use "direct". Note that indirect will fail if the GCS bucket has a retention policy set on it.
{%- if partition_config %}
      {%- set write_method = 'indirect' -%}
{%- else %}
      {% set write_method = 'direct' -%}
{%- endif %}

df.write \
  .mode("overwrite") \
  .format("bigquery") \
  .option("writeMethod", "{{ write_method }}") \
  .option("writeDisposition", 'WRITE_TRUNCATE') \
  {%- if partition_config is not none %}
  {%- if partition_config.data_type | lower in ('date','timestamp','datetime') %}
  .option("partitionField", "{{- partition_config.field -}}") \
  {%- if partition_config.granularity is not none %}

View on GitHub (pinned to 0267ce9170)

Solutions

1. Convert the return value to a pandas DataFrame before returning it from the Python model (e.g. df = other_df.to_pandas()).
2. If using polars, return df.to_pandas(); if using PySpark, return a pyspark.sql.DataFrame directly.
3. Verify pandas is installed in the BigQuery Python model environment (add it to packages.yaml for the model) so pandas_available is True.
4. Return a databricks.koalas DataFrame only if that library is genuinely available; otherwise prefer pandas.
5. Check the model function actually returns a DataFrame, not a list/dict/tuple.

Example fix

# before
def model(dbt, session):
    df = pl.read_parquet('data.parquet')
    return df  # polars -> exception
# after
def model(dbt, session):
    df = pl.read_parquet('data.parquet')
    return df.to_pandas()  # pandas DataFrame is supported

def ensure_supported_df(df):
    import pandas as pd
    if isinstance(df, pd.DataFrame):
        return df
    if hasattr(df, 'to_pandas'):
        return df.to_pandas()
    raise TypeError(f'{type(df)} unsupported; return pandas/Spark/Koalas DataFrame')

def is_supported_df(df) -> bool:
    try:
        import pandas as pd
        if isinstance(df, pd.DataFrame):
            return True
    except ImportError:
        pass
    try:
        from pyspark.sql import DataFrame as SparkDF
        if isinstance(df, SparkDF):
            return True
    except ImportError:
        pass
    try:
        import databricks.koalas
        if isinstance(df, databricks.koalas.frame.DataFrame):
            return True
    except ImportError:
        pass
    return False

try:
    result = runner.invoke(['run', '--select', 'my_python_model'])
except Exception as e:
    if 'is not a supported type for dbt Python materialization' in str(e):
        fix_model_return_type('my_python_model')
    else:
        raise

When it happens

Trigger: A BigQuery Python model returns a type other than pandas.DataFrame, databricks.koalas DataFrame, or pyspark DataFrame — e.g. a polars DataFrame, a list, a dict, or a pyspark.pandas DataFrame when the macro's availability checks didn't detect pandas/pyspark in the submitted job environment.

Common situations: Returning polars/modin/other DataFrame libraries from a Python model; returning a plain list of dicts or a NumPy array; the model imports failing so pandas_available/koalas_available flags are False even though the code looks correct; using pyspark.pandas (not databricks.koalas) which this check doesn't recognize.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• {type(df)} is not a supported type for dbt Python materializ
• {type(df)} is not a supported type for dbt Python materializ
• {type(df)} is not a supported type for dbt Python materializ
• remote_state must be Object
• Argument must be a string

AI-assisted analysis of dbt-labs/dbt-core@0267ce9170 (2026-09-07). Data as JSON: /api/errors/dff299008b54bca9. Report an issue: GitHub.