continuedev/continue · error · Error

Token limit reached. File/range likely too large for this ed

Error message

Token limit reached. File/range likely too large for this edit

What it means

recursiveStream (string variant) counts tokens of streamed chunks against a safe budget (contextLength minus reserve); exceeding it aborts with 'Token limit reached' because the recursive edit output would not fit.

Source

Thrown at core/edit/recursiveStream.ts:55

  // let whiteSpaceAtEndOfBuffer = buffer.match(/\s*$/)?.[0] ?? ""; // attempts at fixing whitespace bug with recursive boundaries

  const injectApplyToken = type === "apply" && shouldInjectApplyToken(llm);
  if (typeof prompt === "string") {
    const finalPrompt = injectApplyToken ? prompt + APPLY_UNIQUE_TOKEN : prompt;

    const generator = llm.streamComplete(finalPrompt, abortController.signal, {
      raw: true,
      prediction: undefined,
      reasoning: false,
    });

    for await (const chunk of generator) {
      yield chunk;
      buffer += chunk;
      totalTokens += countTokens(chunk);

      if (totalTokens >= safeTokens) {
        throw new Error(
          "Token limit reached. File/range likely too large for this edit",
        );
        // const continuationPrompt = `${RECURSIVE_PROMPT}:\n\n${buffer}`;

        // await generator.return(DUD_PROMPT_LOG); // kill the previous generator

        // // TODO - Prediction capabilities lost because of partial input
        // yield* recursiveStream(
        //   llm,
        //   abortController,
        //   continuationPrompt,
        //   undefined,
        //   buffer,
        //   true,
        // ); // Recursively stream the continuation

        // return;
      }

View on GitHub (pinned to 5522c6f44c)

Solutions

Edit a smaller range/selection instead of the whole file
Use a model with larger contextLength
Lower completionOptions.maxTokens reserve or raise context budget
Split the edit into multiple smaller edits

Defensive patterns

Strategy: validation

Validate before calling

const est = countTokens(input);
if (est > llm.contextLength - reserve) { splitIntoSmallerRanges(input); }

Try / catch

catch (e) { if (e.message.startsWith('Token limit reached')) { split the file/range and retry with smaller inputs; } }

Prevention

Estimate token counts before whole-file edits
Prefer range/selection edits for large files

When it happens

Trigger: An edit/stream of a very large file or range where cumulative streamed tokens reach safeTokens before completion.

Common situations: Whole-file rewrites of huge files with a small-context model; maxTokens/completionOptions set too low; long recursive edit loops.

Related errors

AI-assisted analysis of continuedev/continue@5522c6f44c (2026-08-27). Data as JSON: /api/errors/a5e04cab77d4b819. Report an issue: GitHub.