{"record":{"id":"0cd50c6250bafef5","repo":"docling-project/docling","slug":"cannot-convert-csv-with-unknown-delimiter-dialect","errorCode":null,"errorMessage":"Cannot convert csv with unknown delimiter {dialect.delimiter}.","messagePattern":"Cannot convert csv with unknown delimiter (.+?)\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"warning","filePath":"docling/backend/csv_backend.py","lineNumber":63,"sourceCode":"        if isinstance(self.path_or_stream, BytesIO):\n            self.path_or_stream.close()\n        self.path_or_stream = None\n\n    @classmethod\n    def supported_formats(cls) -> Set[InputFormat]:\n        return {InputFormat.CSV}\n\n    def convert(self) -> DoclingDocument:\n        \"\"\"\n        Parses the CSV data into a structured document model.\n        \"\"\"\n\n        # Detect CSV dialect\n        head = self.content.readline()\n        try:\n            dialect: type[csv.Dialect] = csv.Sniffer().sniff(head, \",;\\t|:\")\n            if dialect.delimiter not in {\",\", \";\", \"\\t\", \"|\", \":\"}:\n                raise RuntimeError(\n                    f\"Cannot convert csv with unknown delimiter {dialect.delimiter}.\"\n                )\n            else:\n                _log.info(f'Parsing CSV with delimiter: \"{dialect.delimiter}\"')\n        except csv.Error as e:\n            # Fall back to default commad delimiter (e.g. single-column, insufficient data to detect)\n            _log.info(\n                f\"Could not detect delimiter ({e}), using default comma delimiter\"\n            )\n            dialect = csv.excel\n\n        # Parse CSV\n        self.content.seek(0)\n        result = csv.reader(self.content, dialect=dialect, strict=True)\n        self.csv_data = list(result)\n        _log.info(f\"Detected {len(self.csv_data)} lines\")\n\n        # Parse the CSV into a structured document model","sourceCodeStart":45,"sourceCodeEnd":81,"githubUrl":"https://github.com/docling-project/docling/blob/61d76f1ff3f8428065465889f7b4577da7df704c/docling/backend/csv_backend.py#L45-L81","documentation":"CsvDocumentBackend.convert() runs csv.Sniffer on the first line, restricted to the delimiters , ; \\t | :. If the sniffer returns a delimiter outside that allowed set, the backend raises RuntimeError rather than parsing with an unexpected separator. In practice the sniffer is already restricted to that set, so this guard is defensive; the common path for undetectable files is the csv.Error branch which logs and falls back to the comma dialect instead.","triggerScenarios":"Calling convert() on a CSV whose first line makes csv.Sniffer().sniff return an unexpected delimiter character. Single-column CSVs or very short first lines normally take the csv.Error fallback path (comma, info log) and do NOT raise this.","commonSituations":"Files whose header row is mostly punctuation, causing the sniffer to latch onto an odd character; whitespace-delimited data renamed to .csv; defensively coded pipelines wondering why a weird delimiter works fine (it fell back to comma).","solutions":["Normalize the file to a standard delimiter before conversion (e.g. convert whitespace/other separators to commas).","If the file is single-column, do nothing — the backend already falls back to the comma dialect; the error means the data truly has an unusual active delimiter.","As a last resort, wrap convert() in try/except RuntimeError and re-emit the file after delimiter normalization."],"exampleFix":"# before\nresult = conv.convert(Path('data.csv'))  # sniffer picks unusual delimiter\n\n# after\nfrom io import BytesIO\ntext = Path('data.csv').read_text('utf-8').replace('<odd-delimiter>', ',')\nresult = conv.convert(BytesIO(text.encode('utf-8')))","handlingStrategy":"fallback","validationCode":"import csv, io\n\ndef sniff_delimiter(text_head: str) -> str | None:\n    try:\n        d = csv.Sniffer().sniff(text_head, \",;\\t|:\")\n        return d.delimiter if d.delimiter in {\",\", \";\", \"\\t\", \"|\", \":\"} else None\n    except csv.Error:\n        return None  # backend falls back to comma automatically","typeGuard":null,"tryCatchPattern":"try:\n    result = conv.convert(src)\nexcept RuntimeError as e:\n    if \"unknown delimiter\" in str(e):\n        text = normalize_to_comma(src)  # your own rewriter\n        result = conv.convert(BytesIO(text.encode(\"utf-8\")))\n    else:\n        raise","preventionTips":["Normalize unusual delimiters to commas during ingestion.","Remember single-column/undetectable CSVs fall back to comma automatically — no error.","Only the truly odd active-delimiter case raises; inspect the header line first."],"tags":["csv","delimiter","sniffer","convert"],"backgroundTag":null,"analyzedSha":"61d76f1ff3f8428065465889f7b4577da7df704c","analyzedAt":"2026-08-14T23:53:18.727Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}