{"record":{"id":"419e553487eff64c","repo":"zylon-ai/private-gpt","slug":"error-reading-delimited-file-e","errorCode":null,"errorMessage":"Error reading delimited file: {e}","messagePattern":"Error reading delimited file: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"private_gpt/components/readers/text/delimiter_reader.py","lineNumber":86,"sourceCode":"                    separator_row = \"| \" + \" | \".join(\"-\" for _ in chunk.columns) + \" |\"\n                    markdown_lines.append(header_row)\n                    markdown_lines.append(separator_row)\n                    first_chunk = False\n\n                # Convert chunk to strings and format rows\n                chunk_str = chunk.astype(str)\n                for row in chunk_str.values.tolist():\n                    markdown_lines.append(format_row(row))\n\n            # Join all Markdown lines into a single string.\n            markdown_content = \"\\n\".join(markdown_lines)\n            yield Document(\n                text=markdown_content,\n                extra_info=extra_info if extra_info is not None else {},\n            )\n\n        except Exception as e:\n            raise ValueError(f\"Error reading delimited file: {e}\") from e\n","sourceCodeStart":68,"sourceCodeEnd":87,"githubUrl":"https://github.com/zylon-ai/private-gpt/blob/4a030776a31a901ad80b1bf4d7faa2c1a367efbb/private_gpt/components/readers/text/delimiter_reader.py#L68-L87","documentation":"The delimiter reader wraps its whole parse/emit block in try/except and re-raises any exception as ValueError('Error reading delimited file: ...') chained to the original. It is an aggregation wrapper: the informative part is the inner exception text and its __cause__ (parser errors, encoding errors, dtype problems). Any failure while chunk-reading a delimited file and formatting rows as Markdown surfaces here.","triggerScenarios":"Running the delimiter reader on a .csv/.tsv/.psv file where pandas chunked reading or row formatting raises — malformed lines, undecodable bytes, mixed dtypes in a column, or a wrong delimiter setting.","commonSituations":"Files with embedded stray quotes or stray delimiters; non-UTF-8 encodings (latin-1 exports); Excel exports with BOM/metadata rows; inconsistent column counts; very large files where chunk boundaries expose dtype inference issues.","solutions":["Inspect `exc.__cause__` and the appended original message — it usually names the exact line/byte that failed.","Reproduce the parse directly: `pd.read_csv(path, chunksize=..., sep=<same>)` and apply the fix it suggests (encoding='utf-8-sig', on_bad_lines='skip', dtype=str, quoting=...).","Pre-clean or re-export the file (fix encoding, strip preamble rows) before ingestion.","If occasional bad lines are acceptable, configure the reader to skip them rather than fail the whole document."],"exampleFix":"# before\nfor doc in reader.load_data(file=Path(\"export.csv\")):\n    ...  # ValueError: Error reading delimited file: ...\n\n# after\n# pre-check the file parses cleanly with the same options\npreview = pd.read_csv(\"export.csv\", nrows=100, sep=\",\", encoding=\"utf-8-sig\")\nfor doc in reader.load_data(file=Path(\"export.csv\")):\n    ...","handlingStrategy":"try-catch","validationCode":"import pandas as pd\n\ndef delimited_file_parses(path: str, sep: str) -> bool:\n    try:\n        pd.read_csv(path, sep=sep, nrows=10, encoding=\"utf-8-sig\")\n        return True\n    except Exception:\n        return False","typeGuard":null,"tryCatchPattern":"try:\n    docs = list(reader.load_data(file=path))\nexcept ValueError as e:\n    cause = e.__cause__ or e\n    logger.error(\"Delimited parse failed (%s): %s\", path, cause)\n    quarantine(path)\n    raise","preventionTips":["Read with explicit sep and encoding='utf-8-sig' to avoid common parse failures.","Dry-run pandas.read_csv on a sample before full ingestion of untrusted files.","Keep on_bad_lines policy consistent between pre-check and reader."],"tags":["csv","parsing","ingestion","encoding","wrapped-error"],"backgroundTag":null,"analyzedSha":"4a030776a31a901ad80b1bf4d7faa2c1a367efbb","analyzedAt":"2026-08-15T03:51:26.951Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}