{"record":{"id":"959b1d669478a276","repo":"deepset-ai/haystack","slug":"header-split-levels-must-not-contain-duplicate-val","errorCode":null,"errorMessage":"header_split_levels must not contain duplicate values.","messagePattern":"header_split_levels must not contain duplicate values\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"haystack/components/preprocessors/markdown_header_splitter.py","lineNumber":69,"sourceCode":"        :param split_overlap: The number of overlapping units for each split when using secondary splitting.\n            Defaults to 0.\n        :param split_threshold: The minimum number of units per split when using secondary splitting. Defaults to 0.\n        :param skip_empty_documents: Choose whether to skip documents with empty content. Default is True.\n            Set to False when downstream components in the Pipeline (like LLMDocumentContentExtractor) can extract text\n            from non-textual documents.\n        \"\"\"\n        if header_split_levels is None:\n            header_split_levels = [1, 2, 3, 4, 5, 6]\n\n        if not isinstance(header_split_levels, list) or len(header_split_levels) == 0:\n            raise ValueError(\"header_split_levels must be a non-empty list.\")\n        invalid = [lvl for lvl in header_split_levels if not isinstance(lvl, int) or lvl < 1 or lvl > 6]\n        if invalid:\n            raise ValueError(\n                f\"header_split_levels contains invalid values: {invalid}. All levels must be integers between 1 and 6.\"\n            )\n        if len(header_split_levels) != len(set(header_split_levels)):\n            raise ValueError(\"header_split_levels must not contain duplicate values.\")\n\n        self.page_break_character = page_break_character\n        self.secondary_split = secondary_split\n        self.split_length = split_length\n        self.split_overlap = split_overlap\n        self.split_threshold = split_threshold\n        self.skip_empty_documents = skip_empty_documents\n        self.keep_headers = keep_headers\n        self.header_split_levels = header_split_levels\n        self._header_split_levels_set = set(header_split_levels)\n        self._header_pattern = re.compile(r\"(?m)^(#{1,6}) (.+)$\")  # ATX-style .md-headers\n\n        # Matches fenced code blocks delimited by triple backticks (```) or triple tildes (~~~).\n        # Broken down:\n        #   ^                 - fence must start at the beginning of a line (MULTILINE)\n        #   (?P<fence>`{3,}|~{3,})\n        #                     - named capture group \"fence\": three or more backticks OR three or\n        #                       more tildes. Capturing it allows the closing fence to be matched","sourceCodeStart":51,"sourceCodeEnd":87,"githubUrl":"https://github.com/deepset-ai/haystack/blob/e318778c9bf60a1963e3b5f451359655dd696c30/haystack/components/preprocessors/markdown_header_splitter.py#L51-L87","documentation":"MarkdownHeaderSplitter's __init__ rejects duplicate values in header_split_levels. Duplicates have no semantic effect and would cause redundant checks during splitting, so a ValueError is raised.","triggerScenarios":"Calling MarkdownHeaderSplitter(header_split_levels=[1, 1, 2]) or building the list by concatenation/range where levels repeat (e.g. list(range(1,4))*2).","commonSituations":"Programmatically assembling levels from multiple config sources and merging without deduplication, or copy-pasting level lists in code.","solutions":["Deduplicate the list before construction: header_split_levels=sorted(set(levels)).","Keep only unique values while preserving order: list(dict.fromkeys(levels)).","Remove the duplicated literal from a hand-written list."],"exampleFix":"// before\nMarkdownHeaderSplitter(header_split_levels=[1, 2, 2, 3])\n// after\nMarkdownHeaderSplitter(header_split_levels=sorted(set([1, 2, 2, 3])))  # [1, 2, 3]","handlingStrategy":"validation","validationCode":"levels = list(dict.fromkeys(raw_levels))  # dedupe, preserve order\nassert len(levels) == len(set(levels))","typeGuard":"def has_no_duplicates(levels) -> bool:\n    return isinstance(levels, list) and len(levels) == len(set(levels))","tryCatchPattern":"try:\n    splitter = MarkdownHeaderSplitter(header_split_levels=levels)\nexcept ValueError as e:\n    levels = sorted(set(levels))\n    splitter = MarkdownHeaderSplitter(header_split_levels=levels)","preventionTips":["Deduplicate merged config lists with set() or dict.fromkeys().","Build levels via range(1, 7) slices rather than manual concatenation.","Add a duplicate check in config loading tests."],"tags":["python","validation","duplicates","haystack"],"backgroundTag":"invalid-constructor-argument","analyzedSha":"e318778c9bf60a1963e3b5f451359655dd696c30","analyzedAt":"2026-08-30T11:45:20.711Z","schemaVersion":2},"datasetVersion":"2026-08-30T13:17:10.514Z"}