rohitg00/agentmemory · error · Error

too_many_chunks_skipped: ${skipped}/${chunks.length} chunks

Error message

too_many_chunks_skipped: ${skipped}/${chunks.length} chunks failed to parse after retry

What it means

produceSummaryXml summarizes a session by chunking it and parsing each chunk into a SessionSummary via an LLM. Chunks whose output fails to parse are retried once; if they still fail they are counted as skipped. When the skipped count exceeds MAX_SKIP_RATIO of total chunks, the function throws instead of returning a mostly-missing summary.

Source

Thrown at src/functions/summarize.ts:156

      batch.map(async (chunk, j) => {
        const idx = batchStart + j;
        partialByIdx[idx] = await summarizeChunkWithRetry(
          provider,
          chunk,
          sessionId,
          project,
          idx,
          chunks.length,
        );
      }),
    );
  }

  const skipped = partialByIdx.filter((p) => p === null).length;
  const partials = partialByIdx.filter((p): p is SessionSummary => p !== null);

  if (skipped > Math.floor(chunks.length * MAX_SKIP_RATIO)) {
    throw new Error(
      `too_many_chunks_skipped: ${skipped}/${chunks.length} chunks failed to parse after retry`,
    );
  }
  if (skipped > 0) {
    logger.warn("Summarize chunks partially skipped", {
      sessionId,
      skipped,
      total: chunks.length,
    });
  }

  const reduceInput = partials.map((p) => {
    const originalIdx = partialByIdx.indexOf(p);
    return {
      title: p.title,
      narrative: p.narrative,
      keyDecisions: p.keyDecisions,
      filesModified: p.filesModified,

View on GitHub (pinned to e04ba88819)

Solutions

Check LLM provider health/rate limits and retry the summarize operation once service is restored.
Increase the response token limit / adjust the chunking so each chunk fits and responses are not truncated.
Inspect logger.warn output to identify which chunks failed and re-run with fewer chunks per call.
If recurring, relax MAX_SKIP_RATIO or fix the parser/prompt format mismatch in the summarizer.

Defensive patterns

Strategy: retry

Try / catch

try {
  summary = await sdk.trigger({ function_id: "mem::summarize", payload });
} catch (e) {
  if (String(e.message).startsWith("too_many_chunks_skipped")) {
    await backoff(2000);
    summary = await sdk.trigger({ function_id: "mem::summarize", payload }); // retry after provider recovers
  } else throw e;
}

Prevention

Monitor LLM provider error/rate-limit rates before large batch summarizes.
Keep chunks small enough that responses are never truncated by max_tokens.
Alert on logger.warn 'Summarize chunks partially skipped' — a spike precedes the throw.

When it happens

Trigger: Batch-summarizing many chunks where more than floor(chunks.length * MAX_SKIP_RATIO) chunk summaries come back null after the retry pass, e.g. an LLM endpoint returning malformed/non-XML output or timeouts for a majority of chunks.

Common situations: LLM provider outages or rate limiting causing repeated malformed responses, prompts producing output the XML parser cannot handle (unescaped tags, truncated responses at low max_tokens), or very large sessions producing many chunks that all hit context limits.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of rohitg00/agentmemory@e04ba88819 (2026-08-30). Data as JSON: /api/errors/8aa4aa14638edb79. Report an issue: GitHub.