{"record":{"id":"33ebe844f98d387a","repo":"apache/superset","slug":"error-reading-csv-file","errorCode":null,"errorMessage":"Error reading CSV file","messagePattern":"Error reading CSV file","errorType":"exception","errorClass":"DatabaseUploadFailed","httpStatus":422,"severity":"error","filePath":"superset/commands/database/uploaders/csv_reader.py","lineNumber":471,"sourceCode":"\n            file.seek(0)\n            detected_encoding = CSVReader._detect_encoding(file)\n            if detected_encoding != encoding:\n                kwargs[\"encoding\"] = detected_encoding\n                return CSVReader._read_csv(file, kwargs)\n            raise DatabaseUploadFailed(\n                message=_(\"Parsing error: %(error)s\", error=str(ex))\n            ) from ex\n        except (\n            pd.errors.ParserError,\n            pd.errors.EmptyDataError,\n            ValueError,\n        ) as ex:\n            raise DatabaseUploadFailed(\n                message=_(\"Parsing error: %(error)s\", error=str(ex))\n            ) from ex\n        except Exception as ex:\n            raise DatabaseUploadFailed(_(\"Error reading CSV file\")) from ex\n\n    def file_to_dataframe(self, file: FileStorage) -> pd.DataFrame:\n        \"\"\"\n        Read CSV file into a DataFrame\n\n        :return: pandas DataFrame\n        :throws DatabaseUploadFailed: if there is an error reading the file\n        \"\"\"\n        rows_to_read = self._options.get(\"rows_to_read\")\n        chunk_size = current_app.config.get(\"READ_CSV_CHUNK_SIZE\", 1000)\n\n        use_chunking = rows_to_read is None or rows_to_read > chunk_size * 2\n\n        kwargs = {\n            \"encoding\": self._options.get(\"encoding\", DEFAULT_ENCODING),\n            \"header\": self._options.get(\"header_row\", 0),\n            \"decimal\": self._options.get(\"decimal_character\", \".\"),\n            \"index_col\": self._options.get(\"index_column\"),","sourceCodeStart":453,"sourceCodeEnd":489,"githubUrl":"https://github.com/apache/superset/blob/f4587218dd19d046c3e4d00063e7d27f8a2ed354/superset/commands/database/uploaders/csv_reader.py#L453-L489","documentation":"Catch-all DatabaseUploadFailed ('Error reading CSV file') raised in CSVReader._read_csv for any exception outside the recognized sets (not DatabaseUploadFailed, UnicodeDecodeError, ParserError, EmptyDataError, ValueError). Original exception is chained ('from ex') so server logs show the true cause — commonly OSError, KeyError from bad kwargs, dtype-related TypeError, or chunking/iterator errors in the chunked read path.","triggerScenarios":"pd.read_csv raising OSError (unreadable/revoked stream), TypeError from incompatible options (e.g. dtype mapping problems), MemoryError on very wide/large CSVs, or chunked iteration failures when rows_to_read exceeds READ_CSV_CHUNK_SIZE-based chunking logic.","commonSituations":"Very large uploads exhausting memory before pandas can stream; options combinations the CSV reader builds internally (index_col, header handling) conflicting; uploads where the stream was consumed twice (seek on a non-seekable stream).","solutions":["Check the Superset server log for the chained 'from ex' cause — it identifies the real exception class","Reduce the upload size or use rows_to_read to limit rows and confirm the file parses standalone","For memory pressure, raise worker memory or split the CSV and upload in parts","Reproduce with the same kwargs: pd.read_csv(file, **kwargs) locally"],"exampleFix":"head -n 1000 big.csv > sample.csv   # verify sample parses, then split upload\nsplit -l 500000 big.csv part_  # upload parts sequentially","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"except DatabaseUploadFailed:\n    log.exception(\"csv read failed\")  # 'from ex' chain in logs identifies OSError/TypeError/MemoryError\n    raise","preventionTips":["Split very large CSVs and upload in parts","Limit rows with rows_to_read during trial uploads","Reproduce with the same read_csv kwargs locally"],"tags":["csv","pandas","file-upload","catch-all"],"backgroundTag":null,"analyzedSha":"f4587218dd19d046c3e4d00063e7d27f8a2ed354","analyzedAt":"2026-08-14T22:39:27.425Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}