{"record":{"id":"1085c5253fd1537a","repo":"vercel/ai","slug":"chunking-function-must-return-a-non-empty-string","errorCode":null,"errorMessage":"Chunking function must return a non-empty string.","messagePattern":"Chunking function must return a non-empty string\\.","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"packages/ai/src/generate-text/smooth-stream.ts","lineNumber":71,"sourceCode":"    typeof chunking.segment === 'function'\n  ) {\n    const segmenter = chunking as Intl.Segmenter;\n    detectChunk = (buffer: string) => {\n      if (buffer.length === 0) return null;\n      const iterator = segmenter.segment(buffer)[Symbol.iterator]();\n      const first = iterator.next().value;\n      return first?.segment || null;\n    };\n  } else if (typeof chunking === 'function') {\n    detectChunk = buffer => {\n      const match = chunking(buffer);\n\n      if (match == null) {\n        return null;\n      }\n\n      if (!match.length) {\n        throw new Error(`Chunking function must return a non-empty string.`);\n      }\n\n      if (!buffer.startsWith(match)) {\n        throw new Error(\n          `Chunking function must return a match that is a prefix of the buffer. Received: \"${match}\" expected to start with \"${buffer}\"`,\n        );\n      }\n\n      return match;\n    };\n  } else {\n    const chunkingRegex =\n      typeof chunking === 'string'\n        ? CHUNKING_REGEXPS[chunking]\n        : chunking instanceof RegExp\n          ? chunking\n          : undefined;\n","sourceCodeStart":53,"sourceCodeEnd":89,"githubUrl":"https://github.com/vercel/ai/blob/69428b1f8b037e4d118fb4853428d5c4e620493c/packages/ai/src/generate-text/smooth-stream.ts#L53-L89","documentation":"Plain Error thrown by smoothStream when a custom ChunkDetector function returns an empty string ('' or zero-length match). A chunker must either return null (nothing to flush yet) or a non-empty prefix of the buffer; an empty return is treated as a programming bug in the detector.","triggerScenarios":"Passing `chunking: (buffer) => ...` that returns '' — e.g. slicing with an index computed as 0, regex with optional capture yielding empty, or returning `match?.[0] ?? ''` instead of null.","commonSituations":"Custom detectors written with `buffer.slice(0, n)` where n=0 on first token; using `match[1]` from a regex whose capture group didn't participate; returning empty on whitespace-only buffers.","solutions":["Return `null` instead of an empty string when no chunk should be emitted yet.","Check the detector for paths that can compute a zero-length slice; guard with `if (idx <= 0) return null;`.","Test the detector against edge inputs: '', ' ', first partial token, multi-byte characters."],"exampleFix":"// before\nconst chunking = (buffer: string) => buffer.slice(0, buffer.indexOf(' ') + 1); // '' when no space\n// after\nconst chunking = (buffer: string) => {\n  const idx = buffer.indexOf(' ');\n  return idx === -1 || idx + 1 === 0 ? null : buffer.slice(0, idx + 1);\n};","handlingStrategy":"validation","validationCode":"const detector = (buffer: string) => {\n  const chunk = computeChunk(buffer);\n  if (chunk != null && chunk.length === 0) return null; // never return ''\n  return chunk;\n};","typeGuard":"function isValidDetectorResult(r: string | null): r is string {\n  return r === null || r.length > 0;\n}","tryCatchPattern":"try {\n  result.textStream.pipeThrough(smoothStream({ chunking: detector }).pipeThrough(new TextEncoderStream()));\n} catch (e) {\n  if (e instanceof Error && e.message.includes('non-empty string')) {\n    // fall back to default word chunking\n  } else throw e;\n}","preventionTips":["Return null, never '', from custom detectors when no chunk is ready.","Unit-test detectors with empty, single-char, and whitespace-only buffers.","Avoid slice expressions that can yield length 0 (guard the index)."],"tags":["streaming","smooth-stream","custom-detector"],"backgroundTag":"invalid-chunk-detector-return","analyzedSha":"69428b1f8b037e4d118fb4853428d5c4e620493c","analyzedAt":"2026-08-30T12:32:21.016Z","schemaVersion":2},"datasetVersion":"2026-08-30T13:17:10.514Z"}