{"record":{"id":"2f5b417ca1261334","repo":"stablyai/orca","slug":"benchmark-produced-an-empty-toc-document-generati","errorCode":null,"errorMessage":"benchmark produced an empty TOC; document generation is broken","messagePattern":"benchmark produced an empty TOC; document generation is broken","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"config/scripts/markdown-toc-parse-benchmark.mjs","lineNumber":112,"sourceCode":"  const mid = Math.floor(sorted.length / 2)\n  return sorted.length % 2 === 0 ? (sorted[mid - 1] + sorted[mid]) / 2 : sorted[mid]\n}\n\nfunction measureDoc(headings, parasPerHeading) {\n  const baseDoc = buildDocument(headings, parasPerHeading)\n  // Simulate typing: each content change appends one character so `content`\n  // changes identity every time (matching the memo dependency in the editor).\n  const perChange = []\n  for (let i = 0; i < WARMUP + CONTENT_CHANGES; i += 1) {\n    const doc = `${baseDoc}\\nedit-${i}`\n    const t0 = performance.now()\n    const toc = buildMarkdownTableOfContentsLike(doc)\n    const elapsed = performance.now() - t0\n    if (i >= WARMUP) {\n      perChange.push(elapsed)\n    }\n    if (toc.length === 0) {\n      throw new Error('benchmark produced an empty TOC; document generation is broken')\n    }\n  }\n  const total = perChange.reduce((sum, value) => sum + value, 0)\n  return {\n    bytes: Buffer.byteLength(baseDoc, 'utf8'),\n    headings,\n    perChangeMedianMs: median(perChange),\n    perChangeMaxMs: Math.max(...perChange),\n    burstTotalMs: total\n  }\n}\n\nconst sizes = [\n  { headings: Math.round(DOC_HEADINGS / 8), paras: PARAGRAPHS_PER_HEADING },\n  { headings: Math.round(DOC_HEADINGS / 2), paras: PARAGRAPHS_PER_HEADING },\n  { headings: DOC_HEADINGS, paras: PARAGRAPHS_PER_HEADING }\n]\n","sourceCodeStart":94,"sourceCodeEnd":130,"githubUrl":"https://github.com/stablyai/orca/blob/1136503c6a231a16dce8f921f6fadb63d181e8db/config/scripts/markdown-toc-parse-benchmark.mjs#L94-L130","documentation":"Inside the typing-burst loop, each generated document is parsed with buildMarkdownTableOfContentsLike and the resulting TOC must be non-empty. An empty TOC means the synthetic document no longer contains recognizable headings — i.e. the document generator (DOC_HEADINGS / heading format) is broken, and the benchmark would silently measure zero work.","triggerScenarios":"buildMarkdownTableOfContentsLike(doc) returns [] for the base doc. Happens if the heading emitter produces lines that remark does not treat as headings (wrong leading '#' count, frontmatter swallowing the first heading, remarkGfm/remarkFrontmatter plugin misconfiguration).","commonSituations":"Editing buildMarkdownTableOfContentsLike (e.g. bumping remark versions) changes heading detection; the heading template string is changed to something that is not ATX heading syntax; or remarkFrontmatter is configured with a format that eats the first heading.","solutions":["Print baseDoc to stderr and manually run remark-parse on it to confirm headings are detected","Verify the heading template starts with '# ' (ATX) and is not indented or inside a code fence","Pin remark/remark-gfm/remark-frontmatter versions if a transitive bump changed heading parsing","Cross-check against src/renderer/src/components/editor/markdown-table-of-contents.ts which this mirrors"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"const sample = buildMarkdownTableOfContentsLike(baseDoc)\nif (sample.length === 0) {\n  console.error('baseDoc heading sample:\\n', baseDoc.slice(0, 500))\n  throw new Error('document generator produced no headings before running the burst')\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Print baseDoc to stderr before the burst loop so an empty TOC has context","Pin remark/remark-gfm/remark-frontmatter versions in package.json","Keep buildMarkdownTableOfContentsLike in lockstep with markdown-table-of-contents.ts"],"tags":["benchmark","validation","markdown","remark"],"backgroundTag":null,"analyzedSha":"1136503c6a231a16dce8f921f6fadb63d181e8db","analyzedAt":"2026-08-12T23:15:58.167Z","schemaVersion":2},"datasetVersion":"2026-08-12T23:17:12.415Z"}