{"record":{"id":"5fffe804a3438cee","repo":"agentscope-ai/agentscope","slug":"overlap-must-be-less-than-chunk-size-got-overlap","errorCode":null,"errorMessage":"overlap must be less than chunk_size, got overlap={self.overlap}, chunk_size={self.chunk_size}.","messagePattern":"overlap must be less than chunk_size, got overlap=(.+?), chunk_size=(.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/agentscope/rag/_chunker/_approx_token_chunker.py","lineNumber":63,"sourceCode":"            title=\"Chunk Size\",\n            description=\"Maximum number of approximate tokens per chunk.\",\n        )\n        overlap: int = Field(\n            default=50,\n            ge=0,\n            title=\"Overlap\",\n            description=(\n                \"Number of approximate tokens shared between \"\n                \"consecutive chunks.\"\n            ),\n        )\n\n        @model_validator(mode=\"after\")\n        def _overlap_less_than_chunk_size(\n            self,\n        ) -> \"ApproxTokenChunker.Parameters\":\n            if self.overlap >= self.chunk_size:\n                raise ValueError(\n                    \"overlap must be less than chunk_size, got \"\n                    f\"overlap={self.overlap}, chunk_size={self.chunk_size}.\",\n                )\n            return self\n\n    def __init__(\n        self,\n        parameters: \"ApproxTokenChunker.Parameters | None\" = None,\n        **kwargs: Any,\n    ) -> None:\n        \"\"\"Initialize the approx token chunker.\n\n        Args:\n            parameters (`ApproxTokenChunker.Parameters | None`, optional):\n                The chunker parameters (``chunk_size`` and ``overlap``).\n                Defaults to ``Parameters()`` when not provided.\n            **kwargs (`Any`):\n                Deprecated. ``chunk_size`` and ``overlap`` are still","sourceCodeStart":45,"sourceCodeEnd":81,"githubUrl":"https://github.com/agentscope-ai/agentscope/blob/e90f1c7592896cc95f6e5ee506194f533378247d/src/agentscope/rag/_chunker/_approx_token_chunker.py#L45-L81","documentation":"ApproxTokenChunker's Parameters model validator rejects configurations where overlap >= chunk_size, since overlapping chunks must be strictly smaller than the chunk size to make progress and avoid infinite/duplicate chunking.","triggerScenarios":"Constructing Parameters(chunk_size=200, overlap=200) or overlap > chunk_size; also chunk_size=0 with overlap=0 (0 >= 0).","commonSituations":"Copying chunking configs from RAG tutorials with different defaults; computing overlap as a percentage but passing it as absolute tokens; setting chunk_size equal to overlap by accident.","solutions":["Set overlap strictly less than chunk_size, e.g. chunk_size=512, overlap=50","If computing overlap dynamically, clamp: overlap = min(overlap, chunk_size - 1)","A common safe ratio is overlap ≈ 10–20% of chunk_size"],"exampleFix":"# before\nchunker = ApproxTokenChunker(parameters=Parameters(chunk_size=200, overlap=200))\n\n# after\nchunker = ApproxTokenChunker(parameters=Parameters(chunk_size=512, overlap=64))","handlingStrategy":"validation","validationCode":"overlap = min(overlap, chunk_size - 1)\nparams = ApproxTokenChunker.Parameters(chunk_size=chunk_size, overlap=overlap)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep overlap at 10-20% of chunk_size","Compute overlap from chunk_size programmatically, never hardcode both independently"],"tags":["agentscope","rag","chunking","validation","config"],"backgroundTag":"invalid-chunking-config","analyzedSha":"e90f1c7592896cc95f6e5ee506194f533378247d","analyzedAt":"2026-08-28T18:24:12.087Z","schemaVersion":2},"datasetVersion":"2026-08-28T21:17:43.275Z"}