{"record":{"id":"a5fb27976fe1e336","repo":"apache/superset","slug":"error-reading-columnar-file","errorCode":null,"errorMessage":"Error reading Columnar file","messagePattern":"Error reading Columnar file","errorType":"exception","errorClass":"DatabaseUploadFailed","httpStatus":422,"severity":"error","filePath":"superset/commands/database/uploaders/columnar_reader.py","lineNumber":74,"sourceCode":"    def _read_buffer_to_dataframe(self, buffer: IO[bytes]) -> pd.DataFrame:\n        kwargs: dict[str, Any] = {\n            \"path\": buffer,\n        }\n        if self._options.get(\"columns_read\"):\n            kwargs[\"columns\"] = self._options.get(\"columns_read\")\n        try:\n            return pd.read_parquet(**kwargs)\n        except (\n            pd.errors.ParserError,\n            pd.errors.EmptyDataError,\n            UnicodeDecodeError,\n            ValueError,\n        ) as ex:\n            raise DatabaseUploadFailed(\n                message=_(\"Parsing error: %(error)s\", error=str(ex))\n            ) from ex\n        except Exception as ex:\n            raise DatabaseUploadFailed(_(\"Error reading Columnar file\")) from ex\n\n    @staticmethod\n    def _yield_files(file: FileStorage) -> Generator[IO[bytes], None, None]:\n        \"\"\"\n        Yields files from the provided file. If the file is a zip file, it yields each\n        file within the zip file. If it's a single file, it yields the file itself.\n\n        :param file: The file to yield files from.\n        :return: A generator that yields files.\n        \"\"\"\n        file_suffix = Path(file.filename).suffix\n        if not file_suffix:\n            raise DatabaseUploadFailed(_(\"Unexpected no file extension found\"))\n        file_suffix = file_suffix[1:]  # remove the dot\n        if file_suffix == \"zip\":\n            if not is_zipfile(file):\n                raise DatabaseUploadFailed(_(\"Not a valid ZIP file\"))\n            try:","sourceCodeStart":56,"sourceCodeEnd":92,"githubUrl":"https://github.com/apache/superset/blob/f4587218dd19d046c3e4d00063e7d27f8a2ed354/superset/commands/database/uploaders/columnar_reader.py#L56-L92","documentation":"Catch-all DatabaseUploadFailed ('Error reading Columnar file') raised from ColumnarReader._read_buffer_to_dataframe when pd.read_parquet raises anything outside the recognized set (ParserError, EmptyDataError, UnicodeDecodeError, ValueError). Typical causes are pyarrow exceptions (ArrowInvalid, ArrowIOError), OSError for unreadable buffers, or ImportError when the pyarrow engine is missing. The original exception is chained via 'from ex' so the cause is in the traceback, not the message.","triggerScenarios":"pd.read_parquet with engine='pyarrow' raising pyarrow.lib.ArrowInvalid (e.g. Parquet magic bytes not found), OSError/KeyError from a malformed column list, or ImportError/ModuleNotFoundError when the pyarrow package is not installed in the Superset environment.","commonSituations":"Deployments installed without the 'parquet' extra (pip install superset[parquet] equivalents); files that pass the .parquet extension check but are HTML error pages or JSON payloads; files written by uncommon Parquet writers (e.g. old Spark versions) that the installed pyarrow cannot read.","solutions":["Inspect the chained exception in server logs (the 'from ex' cause) — it names the real failure such as ArrowInvalid or ModuleNotFoundError","If pyarrow is missing, install it in the Superset environment and restart","Verify the first 4 bytes of the file are 'PAR1' (Parquet magic) to confirm it is genuinely a Parquet file","If the file comes from another tool, re-export it with a recent pyarrow/pandas version"],"exampleFix":"python -c \"open('data.parquet','rb').read(4)\"  # expect b'PAR1'\npython -c \"import pyarrow; print(pyarrow.__version__)\"  # confirm engine present","handlingStrategy":"try-catch","validationCode":"def looks_like_parquet(fh) -> bool:\n    fh.seek(0)\n    return fh.read(4) == b'PAR1'","typeGuard":null,"tryCatchPattern":"try:\n    df = reader.file_to_dataframe(file)\nexcept DatabaseUploadFailed:\n    log.exception(\"columnar read failed\")  # chained 'from ex' carries root cause (e.g. ModuleNotFoundError: pyarrow)\n    raise","preventionTips":["Install pyarrow in the Superset environment","Check the PAR1 magic bytes before upload","Regenerate files produced by exotic Parquet writers with recent pyarrow"],"tags":["parquet","pyarrow","file-upload","environment"],"backgroundTag":null,"analyzedSha":"f4587218dd19d046c3e4d00063e7d27f8a2ed354","analyzedAt":"2026-08-14T22:39:27.425Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}