{"record":{"id":"8aa4aa14638edb79","repo":"rohitg00/agentmemory","slug":"too-many-chunks-skipped-skipped-chunks-lengt","errorCode":null,"errorMessage":"too_many_chunks_skipped: ${skipped}/${chunks.length} chunks failed to parse after retry","messagePattern":"too_many_chunks_skipped: (.+?)/(.+?) chunks failed to parse after retry","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"src/functions/summarize.ts","lineNumber":156,"sourceCode":"      batch.map(async (chunk, j) => {\n        const idx = batchStart + j;\n        partialByIdx[idx] = await summarizeChunkWithRetry(\n          provider,\n          chunk,\n          sessionId,\n          project,\n          idx,\n          chunks.length,\n        );\n      }),\n    );\n  }\n\n  const skipped = partialByIdx.filter((p) => p === null).length;\n  const partials = partialByIdx.filter((p): p is SessionSummary => p !== null);\n\n  if (skipped > Math.floor(chunks.length * MAX_SKIP_RATIO)) {\n    throw new Error(\n      `too_many_chunks_skipped: ${skipped}/${chunks.length} chunks failed to parse after retry`,\n    );\n  }\n  if (skipped > 0) {\n    logger.warn(\"Summarize chunks partially skipped\", {\n      sessionId,\n      skipped,\n      total: chunks.length,\n    });\n  }\n\n  const reduceInput = partials.map((p) => {\n    const originalIdx = partialByIdx.indexOf(p);\n    return {\n      title: p.title,\n      narrative: p.narrative,\n      keyDecisions: p.keyDecisions,\n      filesModified: p.filesModified,","sourceCodeStart":138,"sourceCodeEnd":174,"githubUrl":"https://github.com/rohitg00/agentmemory/blob/e04ba88819c365c9acf9d6661ea802143e728bd6/src/functions/summarize.ts#L138-L174","documentation":"produceSummaryXml summarizes a session by chunking it and parsing each chunk into a SessionSummary via an LLM. Chunks whose output fails to parse are retried once; if they still fail they are counted as skipped. When the skipped count exceeds MAX_SKIP_RATIO of total chunks, the function throws instead of returning a mostly-missing summary.","triggerScenarios":"Batch-summarizing many chunks where more than floor(chunks.length * MAX_SKIP_RATIO) chunk summaries come back null after the retry pass, e.g. an LLM endpoint returning malformed/non-XML output or timeouts for a majority of chunks.","commonSituations":"LLM provider outages or rate limiting causing repeated malformed responses, prompts producing output the XML parser cannot handle (unescaped tags, truncated responses at low max_tokens), or very large sessions producing many chunks that all hit context limits.","solutions":["Check LLM provider health/rate limits and retry the summarize operation once service is restored.","Increase the response token limit / adjust the chunking so each chunk fits and responses are not truncated.","Inspect logger.warn output to identify which chunks failed and re-run with fewer chunks per call.","If recurring, relax MAX_SKIP_RATIO or fix the parser/prompt format mismatch in the summarizer."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  summary = await sdk.trigger({ function_id: \"mem::summarize\", payload });\n} catch (e) {\n  if (String(e.message).startsWith(\"too_many_chunks_skipped\")) {\n    await backoff(2000);\n    summary = await sdk.trigger({ function_id: \"mem::summarize\", payload }); // retry after provider recovers\n  } else throw e;\n}","preventionTips":["Monitor LLM provider error/rate-limit rates before large batch summarizes.","Keep chunks small enough that responses are never truncated by max_tokens.","Alert on logger.warn 'Summarize chunks partially skipped' — a spike precedes the throw."],"tags":["llm","parsing","summarization"],"backgroundTag":"batch-partial-failure","analyzedSha":"e04ba88819c365c9acf9d6661ea802143e728bd6","analyzedAt":"2026-08-30T01:07:40.754Z","schemaVersion":2},"datasetVersion":"2026-08-30T03:17:51.788Z"}