apache/superset · error · DatabaseUploadFailed

Error reading Columnar file

Error message

Error reading Columnar file

What it means

Catch-all DatabaseUploadFailed ('Error reading Columnar file') raised from ColumnarReader._read_buffer_to_dataframe when pd.read_parquet raises anything outside the recognized set (ParserError, EmptyDataError, UnicodeDecodeError, ValueError). Typical causes are pyarrow exceptions (ArrowInvalid, ArrowIOError), OSError for unreadable buffers, or ImportError when the pyarrow engine is missing. The original exception is chained via 'from ex' so the cause is in the traceback, not the message.

Source

Thrown at superset/commands/database/uploaders/columnar_reader.py:74

    def _read_buffer_to_dataframe(self, buffer: IO[bytes]) -> pd.DataFrame:
        kwargs: dict[str, Any] = {
            "path": buffer,
        }
        if self._options.get("columns_read"):
            kwargs["columns"] = self._options.get("columns_read")
        try:
            return pd.read_parquet(**kwargs)
        except (
            pd.errors.ParserError,
            pd.errors.EmptyDataError,
            UnicodeDecodeError,
            ValueError,
        ) as ex:
            raise DatabaseUploadFailed(
                message=_("Parsing error: %(error)s", error=str(ex))
            ) from ex
        except Exception as ex:
            raise DatabaseUploadFailed(_("Error reading Columnar file")) from ex

    @staticmethod
    def _yield_files(file: FileStorage) -> Generator[IO[bytes], None, None]:
        """
        Yields files from the provided file. If the file is a zip file, it yields each
        file within the zip file. If it's a single file, it yields the file itself.

        :param file: The file to yield files from.
        :return: A generator that yields files.
        """
        file_suffix = Path(file.filename).suffix
        if not file_suffix:
            raise DatabaseUploadFailed(_("Unexpected no file extension found"))
        file_suffix = file_suffix[1:]  # remove the dot
        if file_suffix == "zip":
            if not is_zipfile(file):
                raise DatabaseUploadFailed(_("Not a valid ZIP file"))
            try:

View on GitHub (pinned to f4587218dd)

Solutions

1. Inspect the chained exception in server logs (the 'from ex' cause) — it names the real failure such as ArrowInvalid or ModuleNotFoundError
2. If pyarrow is missing, install it in the Superset environment and restart
3. Verify the first 4 bytes of the file are 'PAR1' (Parquet magic) to confirm it is genuinely a Parquet file
4. If the file comes from another tool, re-export it with a recent pyarrow/pandas version

Example fix

python -c "open('data.parquet','rb').read(4)"  # expect b'PAR1'
python -c "import pyarrow; print(pyarrow.__version__)"  # confirm engine present

def looks_like_parquet(fh) -> bool:
    fh.seek(0)
    return fh.read(4) == b'PAR1'

try:
    df = reader.file_to_dataframe(file)
except DatabaseUploadFailed:
    log.exception("columnar read failed")  # chained 'from ex' carries root cause (e.g. ModuleNotFoundError: pyarrow)
    raise

When it happens

Trigger: pd.read_parquet with engine='pyarrow' raising pyarrow.lib.ArrowInvalid (e.g. Parquet magic bytes not found), OSError/KeyError from a malformed column list, or ImportError/ModuleNotFoundError when the pyarrow package is not installed in the Superset environment.

Common situations: Deployments installed without the 'parquet' extra (pip install superset[parquet] equivalents); files that pass the .parquet extension check but are HTML error pages or JSON payloads; files written by uncommon Parquet writers (e.g. old Spark versions) that the installed pyarrow cannot read.

Related errors

• Parsing error: %(error)s
• Unexpected no file extension found
• Not a valid ZIP file
• str(ex)
• ZIP file contains multiple file types

AI-assisted analysis of apache/superset@f4587218dd (2026-08-14). Data as JSON: /api/errors/a5fb27976fe1e336. Report an issue: GitHub.