{"record":{"id":"89df778d2f0920ab","repo":"mastra-ai/mastra","slug":"chunk-overlap-overlap-must-be-smaller-than-ch","errorCode":null,"errorMessage":"Chunk overlap (${overlap}) must be smaller than chunk size (${maxSize}).","messagePattern":"Chunk overlap \\((.+?)\\) must be smaller than chunk size \\((.+?)\\)\\.","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"packages/rag/src/document/transformers/text.ts","lineNumber":24,"sourceCode":"\nexport abstract class TextTransformer implements Transformer {\n  protected maxSize: number;\n  protected overlap: number;\n  protected lengthFunction: (text: string) => number;\n  protected separatorPosition?: 'start' | 'end';\n  protected addStartIndex: boolean;\n  protected stripWhitespace: boolean;\n\n  constructor({\n    maxSize = 4000,\n    overlap = 200,\n    lengthFunction = (text: string) => text.length,\n    separatorPosition,\n    addStartIndex = false,\n    stripWhitespace = true,\n  }: BaseChunkOptions) {\n    if (overlap >= maxSize) {\n      throw new Error(`Chunk overlap (${overlap}) must be smaller than chunk size (${maxSize}).`);\n    }\n    this.maxSize = maxSize;\n    this.overlap = overlap;\n    this.lengthFunction = lengthFunction;\n    this.separatorPosition = separatorPosition;\n    this.addStartIndex = addStartIndex;\n    this.stripWhitespace = stripWhitespace;\n  }\n\n  setAddStartIndex(value: boolean): void {\n    this.addStartIndex = value;\n  }\n\n  abstract splitText({ text }: { text: string }): string[];\n\n  createDocuments(texts: string[], metadatas?: Record<string, any>[]): Document[] {\n    const _metadatas = metadatas || Array(texts.length).fill({});\n    const documents: Document[] = [];","sourceCodeStart":6,"sourceCodeEnd":42,"githubUrl":"https://github.com/mastra-ai/mastra/blob/75dd419e613fe9c39f846ffc500716141b74fda6/packages/rag/src/document/transformers/text.ts#L6-L42","documentation":"TextTransformer's constructor validates that the overlap between adjacent chunks is strictly smaller than the maximum chunk size. If overlap >= maxSize, chunking is impossible (chunks could never advance), so the constructor refuses to build an invalid transformer. This is a fail-fast configuration check.","triggerScenarios":"new TextTransformer({ maxSize: 500, overlap: 500 }) or any call where the overlap option is greater than or equal to maxSize. Note the default maxSize is typically smaller than a passed overlap too, e.g. maxSize left at default while overlap is set high.","commonSituations":"Copy-pasting chunk configs from another library where overlap was expressed as a ratio (e.g. 0.2) and not noticing the default maxSize is small; switching from characters to tokens without rescaling sizes; off-by-one using overlap equal to size thinking it means contiguous chunks.","solutions":["Set overlap strictly less than maxSize, e.g. maxSize: 1000, overlap: 200.","Check that you are not passing a fraction (0.1–0.5) as overlap; convert it to an absolute number of characters/tokens first.","Verify a custom lengthFunction isn't inflating measured sizes; the comparison uses raw numeric values, but downstream chunking uses the function."],"exampleFix":"// before\nnew TextTransformer({ maxSize: 512, overlap: 512 });\n// after\nnew TextTransformer({ maxSize: 512, overlap: 128 });","handlingStrategy":"validation","validationCode":"const maxSize = 512, overlap = 128;\nif (overlap >= maxSize) throw new Error(`overlap (${overlap}) must be < maxSize (${maxSize})`);","typeGuard":"function isValidChunkConfig(o: { maxSize: number; overlap: number }): boolean {\n  return Number.isInteger(o.maxSize) && o.maxSize > 0 && o.overlap >= 0 && o.overlap < o.maxSize;\n}","tryCatchPattern":"try {\n  const splitter = new TextTransformer({ maxSize: 512, overlap: 128 });\n} catch (e) {\n  if ((e as Error).message.includes('must be smaller than chunk size')) {\n    console.error('Fix chunk config: overlap must be < maxSize');\n  }\n  throw e;\n}","preventionTips":["Always set overlap to a fraction of maxSize (e.g. 10–20%) as an absolute number, never a ratio.","Never set overlap equal to maxSize; the check is strict (>).","Centralize chunking config in one typed constant/object so ratios can't leak in.","When switching length functions (chars vs tokens), recheck both maxSize and overlap scales."],"tags":["rag","chunking","configuration","validation"],"backgroundTag":"invalid-chunk-parameters","analyzedSha":"75dd419e613fe9c39f846ffc500716141b74fda6","analyzedAt":"2026-08-30T00:15:31.844Z","schemaVersion":2},"datasetVersion":"2026-08-30T03:17:51.788Z"}