{"record":{"id":"7330c9c7eb7fdaa5","repo":"tobi/qmd","slug":"qmd-ast-parse-failed-for-filepath-falling-ba","errorCode":null,"errorMessage":"[qmd] AST parse failed for ${filepath}, falling back to regex: ${err instanceof Error ? err.message : err}","messagePattern":"\\[qmd\\] AST parse failed for (.+?), falling back to regex: (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"src/ast.ts","lineNumber":320,"sourceCode":"    const seen = new Map<number, BreakPoint>();\n\n    for (const cap of captures) {\n      const pos = cap.node.startIndex;\n      const score = SCORE_MAP[cap.name] ?? 20;\n      const type = `ast:${cap.name}`;\n\n      const existing = seen.get(pos);\n      if (!existing || score > existing.score) {\n        seen.set(pos, { pos, score, type });\n      }\n    }\n\n    tree.delete();\n    parser.delete();\n\n    return Array.from(seen.values()).sort((a, b) => a.pos - b.pos);\n  } catch (err) {\n    console.warn(`[qmd] AST parse failed for ${filepath}, falling back to regex: ${err instanceof Error ? err.message : err}`);\n    return [];\n  }\n}\n\n// =============================================================================\n// Health / Status\n// =============================================================================\n\n/**\n * Check which tree-sitter grammars are available.\n * Returns a status object for each supported language.\n */\nexport async function getASTStatus(): Promise<{\n  available: boolean;\n  languages: { language: SupportedLanguage; available: boolean; error?: string }[];\n}> {\n  const languages: { language: SupportedLanguage; available: boolean; error?: string }[] = [];\n","sourceCodeStart":302,"sourceCodeEnd":338,"githubUrl":"https://github.com/tobi/qmd/blob/dbfd0b4736aeaf761d1a16ca8e424f071df8feb9/src/ast.ts#L302-L338","documentation":"getASTBreakPoints parses a file with tree-sitter to find function/class boundaries for chunking; if parsing throws, it warns '[qmd] AST parse failed for {filepath}, falling back to regex' and returns an empty breakpoint list. The caller then chunks the file with the regex strategy, so the file is still indexed — only the AST-aware boundaries are lost for that one file.","triggerScenarios":"Indexing a .ts/.js/.py/.go/.rs file with AST chunking where tree-sitter's parser throws on the content: pathological or deeply nested code, files with unusual encodings/BOMs or invalid bytes, extremely large files that trip parser limits, or a stale grammar producing internal errors for that syntax.","commonSituations":"Minified or machine-generated files with extremely long lines; files with mixed encodings or stray NUL bytes after a bad merge; a corrupted source file; grammar edge cases after a tree-sitter upgrade. The warning is per-file and benign unless it affects many files.","solutions":["Treat it as a soft warning first: the file still gets indexed via regex chunking; only act if search quality for that file matters","Check the file for encoding problems: `file -i <path>`, remove BOM/NUL bytes, and confirm it is valid UTF-8","If the file is minified/generated, exclude it from AST chunking (use the regex strategy for that path or exclude it from the collection)","If many files fail after a dependency upgrade, refresh/pin the tree-sitter grammar package (see the related grammar-load warning)"],"exampleFix":"# before\nqmd collection add . --name src --chunk-strategy auto  # warns on gen/file.ts\n\n# after: keep generated code out of AST chunking\nqmd collection add . --name src --chunk-strategy auto --mask 'src/**/*.ts,!src/gen/**'","handlingStrategy":"fallback","validationCode":"import { readFileSync } from 'node:fs';\n\nfunction isSafeForAstParse(filepath: string): boolean {\n  const buf = readFileSync(filepath);\n  if (buf.includes(0)) return false;                 // NUL bytes\n  if (buf.length > 2 * 1024 * 1024) return false;    // oversized/generated files\n  const s = buf.toString('utf8');\n  if (s.includes('\\uFFFD')) return false;            // invalid UTF-8\n  return true;\n}","typeGuard":"function looksParseableSource(text: string): boolean {\n  return !text.includes('\\u0000') && text.length < 1_000_000 && !text.includes('\\uFFFD');\n}","tryCatchPattern":null,"preventionTips":["Keep generated/minified files out of AST-chunked collections via mask exclusions","Normalize file encodings to UTF-8 without BOM before committing","Treat the warning as informational: regex chunking still indexes the file; only investigate if many files warn","After upgrading tree-sitter grammars, re-run qmd update to re-chunk previously failed files"],"tags":["tree-sitter","parse-error","ast-chunking","fallback","encoding"],"backgroundTag":"tree-sitter-parse-failure","analyzedSha":"dbfd0b4736aeaf761d1a16ca8e424f071df8feb9","analyzedAt":"2026-08-28T18:07:46.628Z","schemaVersion":2},"datasetVersion":"2026-08-28T21:17:43.275Z"}