{"record":{"id":"5d5f14e022316d68","repo":"zylon-ai/private-gpt","slug":"chunk-size-must-be-greater-than-0","errorCode":null,"errorMessage":"Chunk size must be greater than 0.","messagePattern":"Chunk size must be greater than 0\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"private_gpt/components/markdown/markdown_helper.py","lineNumber":160,"sourceCode":"        index, lines = data\n\n        processed = []\n        for line in lines:\n            processed.append(MarkdownHelper._safe_sanity_data(line))\n\n        return index, processed\n\n    @staticmethod\n    def sanitize_markdown(\n        markdown: str, chunk_size: int | None = 256, max_workers: int | None = None\n    ) -> str:\n        \"\"\"Sanitize markdown content to fix common formatting issues.\"\"\"\n        if not markdown:\n            return markdown\n\n        chunk_size = chunk_size or len(markdown)\n        if chunk_size < 1:\n            raise ValueError(\"Chunk size must be greater than 0.\")\n\n        max_workers = max_workers or 1\n        max_workers = min(max_workers, len(markdown) // chunk_size)\n        max_workers = max(1, max_workers)\n\n        # Split into chunks while trying to preserve line boundaries\n        lines = markdown.splitlines(keepends=True)\n        batches: list[list[str]] = list(iter_batch(lines, chunk_size))\n\n        # Create pool of workers\n        processed_chunks: list[tuple[int, list[str]]] = []\n        if max_workers == 1:\n            for index, batch in enumerate(batches):\n                _, processed = MarkdownHelper._sanity_markdown_batches((index, batch))\n                processed_chunks.append((index, processed))\n        else:\n            with ThreadPoolExecutor(max_workers=max_workers) as executor:\n                for index, batch in executor.map(","sourceCodeStart":142,"sourceCodeEnd":178,"githubUrl":"https://github.com/zylon-ai/private-gpt/blob/4a030776a31a901ad80b1bf4d7faa2c1a367efbb/private_gpt/components/markdown/markdown_helper.py#L142-L178","documentation":"Raised by MarkdownHelper.sanitize_markdown when the effective chunk_size is below 1. Note the coercion just above: chunk_size = chunk_size or len(markdown), so None/0 fall back to the document length (safe for non-empty input); the only way to reach the error is passing a negative chunk_size explicitly. The value matters because the markdown is split into line batches of chunk_size for parallel sanitizing.","triggerScenarios":"Calling sanitize_markdown(markdown, chunk_size=-5); computing chunk_size dynamically (e.g. len(text) // workers) and passing a negative result; configuration where a negative chunk size is read from settings/UI.","commonSituations":"Derived chunk sizes from division that can go negative; user-supplied settings validated only for presence, not sign; copy-paste of a size constant with the wrong sign.","solutions":["Pass a positive chunk_size (typical 256) or None to let it default to the full document in one chunk.","Clamp computed values: chunk_size = max(1, computed).","Validate the chunk-size setting at load time (must be a positive int)."],"exampleFix":"# before\nout = MarkdownHelper.sanitize_markdown(md, chunk_size=len(md) // num_workers)  # can be negative\n\n# after\nout = MarkdownHelper.sanitize_markdown(md, chunk_size=max(1, len(md) // num_workers))","handlingStrategy":"validation","validationCode":"chunk_size = max(1, chunk_size) if chunk_size is not None else None\nMarkdownHelper.sanitize_markdown(md, chunk_size=chunk_size)","typeGuard":"def is_valid_chunk_size(n: Any) -> bool:\n    return n is None or (isinstance(n, int) and n >= 1)","tryCatchPattern":"try:\n    out = MarkdownHelper.sanitize_markdown(md, chunk_size=cs)\nexcept ValueError:\n    out = MarkdownHelper.sanitize_markdown(md)  # default single chunk","preventionTips":["Clamp every computed chunk size with max(1, value).","Validate chunk-size settings as positive ints in the config schema."],"tags":["validation","markdown","ingestion","configuration"],"backgroundTag":null,"analyzedSha":"4a030776a31a901ad80b1bf4d7faa2c1a367efbb","analyzedAt":"2026-08-15T03:51:26.951Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}