{"record":{"id":"da3a8936bf0801dd","repo":"deepset-ai/haystack","slug":"overlap-cannot-be-greater-than-or-equal-to-the-chu","errorCode":null,"errorMessage":"Overlap cannot be greater than or equal to the chunk size.","messagePattern":"Overlap cannot be greater than or equal to the chunk size\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"haystack/components/preprocessors/recursive_splitter.py","lineNumber":119,"sourceCode":"        \"\"\"\n        Warm up the sentence tokenizer and tiktoken tokenizer if needed.\n        \"\"\"\n        if self._is_warmed_up:\n            return\n        if \"sentence\" in self.separators:\n            self.nltk_tokenizer = self._get_custom_sentence_tokenizer(self.sentence_splitter_params)\n        if self.split_units == \"token\":\n            tiktoken_imports.check()\n            self.tiktoken_tokenizer = tiktoken.get_encoding(\"o200k_base\")\n        self._is_warmed_up = True\n\n    def _check_params(self) -> None:\n        if self.split_length < 1:\n            raise ValueError(\"Split length must be at least 1 character.\")\n        if self.split_overlap < 0:\n            raise ValueError(\"split_overlap must be greater than or equal to 0.\")\n        if self.split_overlap >= self.split_length:\n            raise ValueError(\"Overlap cannot be greater than or equal to the chunk size.\")\n        if not all(isinstance(separator, str) for separator in self.separators):\n            raise ValueError(\"All separators must be strings.\")\n\n    @staticmethod\n    def _get_custom_sentence_tokenizer(sentence_splitter_params: dict[str, Any]) -> Any:\n        from haystack.components.preprocessors.sentence_tokenizer import SentenceSplitter\n\n        return SentenceSplitter(**sentence_splitter_params)\n\n    def _split_chunk(self, current_chunk: str) -> tuple[str, str]:\n        \"\"\"\n        Splits a chunk based on the split_length and split_units attribute.\n\n        :param current_chunk: The current chunk to be split.\n        :returns:\n            A tuple containing the current chunk and the remaining chunk.\n        \"\"\"\n        if self.split_units == \"word\":","sourceCodeStart":101,"sourceCodeEnd":137,"githubUrl":"https://github.com/deepset-ai/haystack/blob/e318778c9bf60a1963e3b5f451359655dd696c30/haystack/components/preprocessors/recursive_splitter.py#L101-L137","documentation":"RecursiveSplitter enforces split_overlap < split_length. If overlap equals or exceeds the chunk size, windows would not advance (infinite loop) or would produce degenerate output, so __init__ rejects it.","triggerScenarios":"RecursiveSplitter(split_length=100, split_overlap=100) or overlap > length; changing split_length downward in config without lowering overlap; swapped parameter values.","commonSituations":"Tuning chunk sizes where overlap was set as a fraction of an older, larger split_length; copy-paste between splitter configs with different scales.","solutions":["Ensure split_overlap < split_length, e.g. keep overlap around 10-20% of split_length.","Recompute overlap whenever split_length changes: overlap = min(overlap, split_length - 1).","Set split_overlap=0 if overlap is not needed."],"exampleFix":"// before\nsplitter = RecursiveSplitter(split_length=100, split_overlap=100)\n// after\nsplitter = RecursiveSplitter(split_length=100, split_overlap=20)","handlingStrategy":"validation","validationCode":"if split_overlap >= split_length:\n    raise ValueError(f'split_overlap ({split_overlap}) must be < split_length ({split_length})')","typeGuard":"def are_valid_chunk_params(length: int, overlap: int) -> bool:\n    return isinstance(length, int) and isinstance(overlap, int) and length >= 1 and 0 <= overlap < length","tryCatchPattern":"try:\n    splitter = RecursiveSplitter(split_length=length, split_overlap=overlap)\nexcept ValueError as e:\n    logger.error('Invalid chunk params: %s', e)\n    splitter = RecursiveSplitter(split_length=length, split_overlap=min(overlap, length - 1))","preventionTips":["Express overlap as a fraction: overlap = int(split_length * 0.15).","Recompute overlap whenever split_length changes.","Centralize splitter construction in one factory that enforces invariants."],"tags":["python","validation","constructor-argument","haystack"],"backgroundTag":"invalid-parameter-value","analyzedSha":"e318778c9bf60a1963e3b5f451359655dd696c30","analyzedAt":"2026-08-30T11:45:20.711Z","schemaVersion":2},"datasetVersion":"2026-08-30T13:17:10.514Z"}