{"record":{"id":"dff299008b54bca9","repo":"dbt-labs/dbt-core","slug":"type-df-is-not-a-supported-type-for-dbt-python","errorCode":null,"errorMessage":"{type(df)} is not a supported type for dbt Python materialization","messagePattern":"(.+?) is not a supported type for dbt Python materialization","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"crates/dbt-loader/src/dbt_macro_assets/dbt-bigquery/macros/materializations/table.sql","lineNumber":114,"sourceCode":"# since they know how to convert pandas DataFrames better than `spark.createDataFrame(df)`\n# and converting from pandas-on-Spark to Spark DataFrame has no overhead\nif pyspark_pandas_api_available and pandas_available and isinstance(df, pandas.core.frame.DataFrame):\n  df = pyspark.pandas.frame.DataFrame(df)\nelif koalas_available and pandas_available and isinstance(df, pandas.core.frame.DataFrame):\n  df = databricks.koalas.frame.DataFrame(df)\n\n# convert to pyspark.sql.dataframe.DataFrame\nif isinstance(df, pyspark.sql.dataframe.DataFrame):\n  pass  # since it is already a Spark DataFrame\nelif pyspark_pandas_api_available and isinstance(df, pyspark.pandas.frame.DataFrame):\n  df = df.to_spark()\nelif koalas_available and isinstance(df, databricks.koalas.frame.DataFrame):\n  df = df.to_spark()\nelif pandas_available and isinstance(df, pandas.core.frame.DataFrame):\n  df = spark.createDataFrame(df)\nelse:\n  msg = f\"{type(df)} is not a supported type for dbt Python materialization\"\n  raise Exception(msg)\n\n# For writeMethod we need to use \"indirect\" if materializing a partitioned table\n# otherwise we can use \"direct\". Note that indirect will fail if the GCS bucket has a retention policy set on it.\n{%- if partition_config %}\n      {%- set write_method = 'indirect' -%}\n{%- else %}\n      {% set write_method = 'direct' -%}\n{%- endif %}\n\ndf.write \\\n  .mode(\"overwrite\") \\\n  .format(\"bigquery\") \\\n  .option(\"writeMethod\", \"{{ write_method }}\") \\\n  .option(\"writeDisposition\", 'WRITE_TRUNCATE') \\\n  {%- if partition_config is not none %}\n  {%- if partition_config.data_type | lower in ('date','timestamp','datetime') %}\n  .option(\"partitionField\", \"{{- partition_config.field -}}\") \\\n  {%- if partition_config.granularity is not none %}","sourceCodeStart":96,"sourceCodeEnd":132,"githubUrl":"https://github.com/dbt-labs/dbt-core/blob/0267ce9170576975b76b64ce856b2e5848e96617/crates/dbt-loader/src/dbt_macro_assets/dbt-bigquery/macros/materializations/table.sql#L96-L132","documentation":"Raised by the dbt-bigquery Python model materialization macro when the Python model's returned DataFrame is neither a supported Spark, Koalas, nor pandas DataFrame. The macro inspects the df returned by the model's Python code and only knows how to convert pandas (via spark.createDataFrame) and Koalas (via to_spark); any other type (or a df that failed to import/convert) hits this exception.","triggerScenarios":"A BigQuery Python model returns a type other than pandas.DataFrame, databricks.koalas DataFrame, or pyspark DataFrame — e.g. a polars DataFrame, a list, a dict, or a pyspark.pandas DataFrame when the macro's availability checks didn't detect pandas/pyspark in the submitted job environment.","commonSituations":"Returning polars/modin/other DataFrame libraries from a Python model; returning a plain list of dicts or a NumPy array; the model imports failing so pandas_available/koalas_available flags are False even though the code looks correct; using pyspark.pandas (not databricks.koalas) which this check doesn't recognize.","solutions":["Convert the return value to a pandas DataFrame before returning it from the Python model (e.g. df = other_df.to_pandas()).","If using polars, return df.to_pandas(); if using PySpark, return a pyspark.sql.DataFrame directly.","Verify pandas is installed in the BigQuery Python model environment (add it to packages.yaml for the model) so pandas_available is True.","Return a databricks.koalas DataFrame only if that library is genuinely available; otherwise prefer pandas.","Check the model function actually returns a DataFrame, not a list/dict/tuple."],"exampleFix":"# before\ndef model(dbt, session):\n    df = pl.read_parquet('data.parquet')\n    return df  # polars -> exception\n# after\ndef model(dbt, session):\n    df = pl.read_parquet('data.parquet')\n    return df.to_pandas()  # pandas DataFrame is supported","handlingStrategy":"type-guard","validationCode":"def ensure_supported_df(df):\n    import pandas as pd\n    if isinstance(df, pd.DataFrame):\n        return df\n    if hasattr(df, 'to_pandas'):\n        return df.to_pandas()\n    raise TypeError(f'{type(df)} unsupported; return pandas/Spark/Koalas DataFrame')","typeGuard":"def is_supported_df(df) -> bool:\n    try:\n        import pandas as pd\n        if isinstance(df, pd.DataFrame):\n            return True\n    except ImportError:\n        pass\n    try:\n        from pyspark.sql import DataFrame as SparkDF\n        if isinstance(df, SparkDF):\n            return True\n    except ImportError:\n        pass\n    try:\n        import databricks.koalas\n        if isinstance(df, databricks.koalas.frame.DataFrame):\n            return True\n    except ImportError:\n        pass\n    return False","tryCatchPattern":"try:\n    result = runner.invoke(['run', '--select', 'my_python_model'])\nexcept Exception as e:\n    if 'is not a supported type for dbt Python materialization' in str(e):\n        fix_model_return_type('my_python_model')\n    else:\n        raise","preventionTips":["Always return a pandas or Spark DataFrame from BigQuery Python models.","Convert polars/modin/other DataFrames with .to_pandas() before returning.","List pandas in the model's packages.yaml so it is installed in the job environment.","Add a unit test asserting the model's return type is a pandas DataFrame.","Never return raw lists/dicts/NumPy arrays from Python models."],"tags":["dbt","bigquery","python-model","dataframe","type-mismatch"],"backgroundTag":"type-mismatch","analyzedSha":"0267ce9170576975b76b64ce856b2e5848e96617","analyzedAt":"2026-09-07T21:53:39.732Z","contentChangedAt":"2026-09-07T21:53:39.732Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}