{"record":{"id":"ba34369766a3aa40","repo":"mastra-ai/mastra","slug":"tokenlimiterprocessor-system-messages-alone-excee","errorCode":null,"errorMessage":"TokenLimiterProcessor: System messages alone exceed token limit. Requests cannot be completed by removing system messages.","messagePattern":"TokenLimiterProcessor: System messages alone exceed token limit\\. Requests cannot be completed by removing system messages\\.","errorType":"exception","errorClass":"TripWire","httpStatus":null,"severity":"error","filePath":"packages/core/src/processors/processors/token-limiter.ts","lineNumber":174,"sourceCode":"      throw new TripWire('TokenLimiterProcessor: No messages to process. Cannot send LLM a request with no messages.', {\n        retry: false,\n      });\n    }\n\n    // Budget against the full system message set that will reach the model\n    // (untagged + tagged buckets), not just the untagged view exposed via args.\n    const allSystemMessages = messageList.getAllSystemMessages();\n    let systemTokens = 0;\n    for (const msg of allSystemMessages) {\n      systemTokens += await this.countCoreSystemMessageTokens(msg);\n    }\n\n    const limit = this.maxTokens;\n\n    // If system messages alone exceed the token limit (accounting for conversation overhead),\n    // throw TripWire - can't send LLM a request with only system messages\n    if (systemTokens + TokenLimiterProcessor.TOKENS_PER_CONVERSATION >= limit) {\n      throw new TripWire(\n        'TokenLimiterProcessor: System messages alone exceed token limit. Requests cannot be completed by removing system messages.',\n        { retry: false, metadata: { systemTokens, limit } },\n      );\n    }\n\n    // Calculate remaining budget for non-system messages (accounting for conversation overhead)\n    const remainingBudget = limit - systemTokens - TokenLimiterProcessor.TOKENS_PER_CONVERSATION;\n\n    // Process non-system messages in reverse order (newest first)\n    const messagesToKeep: MastraDBMessage[] = [];\n    let currentTokens = 0;\n\n    // Iterate through messages in reverse to prioritize recent messages\n    for (let i = messages.length - 1; i >= 0; i--) {\n      const message = messages[i];\n      if (!message) continue;\n\n      const messageTokens = await this.countInputMessageTokens(message);","sourceCodeStart":156,"sourceCodeEnd":192,"githubUrl":"https://github.com/mastra-ai/mastra/blob/75dd419e613fe9c39f846ffc500716141b74fda6/packages/core/src/processors/processors/token-limiter.ts#L156-L192","documentation":"TokenLimiterProcessor counts tokens for system messages plus a fixed conversation overhead (TOKENS_PER_CONVERSATION); if that sum already meets or exceeds maxTokens, no request containing the system messages could ever fit. Since system messages must be sent for the LLM to work, the processor throws a TripWire with retry:false to fail fast instead of producing an invalid request.","triggerScenarios":"processInputStep (via runStep) computes systemTokens from messageList.getAllSystemMessages() and systemTokens + TOKENS_PER_CONVERSATION >= this.maxTokens. Happens when the processor is constructed with a maxTokens smaller than the token count of the system prompt plus overhead.","commonSituations":"Configuring TokenLimiterProcessor({ maxTokens }) with a value tuned only for user messages while the agent has a large system prompt/instructions; shrinking maxTokens after growing memory instructions; misreading maxTokens as the non-system budget.","solutions":["Increase maxTokens so it exceeds system-prompt tokens plus TOKENS_PER_CONVERSATION overhead.","Shorten the system prompt / instructions (trim or move static content out of system messages).","Remove the TokenLimiterProcessor or raise the limit for that agent if the context window genuinely allows it.","Measure actual system token count (countTokens on the system message) and set maxTokens with headroom above it."],"exampleFix":"// before\nnew TokenLimiterProcessor({ maxTokens: 1000 }); // system prompt is 1200 tokens\n// after\nnew TokenLimiterProcessor({ maxTokens: 8000 }); // > systemTokens + overhead","handlingStrategy":"validation","validationCode":"const processor = new TokenLimiterProcessor({ maxTokens });\nconst systemText = messages.filter(m => m.role === 'system' && typeof m.content === 'string').map(m => m.content).join('');\nif (processor.countTokens(systemText) + 1000 >= maxTokens) {\n  throw new Error('maxTokens must exceed system prompt tokens + conversation overhead');\n}","typeGuard":null,"tryCatchPattern":"try {\n  await agent.generate(input);\n} catch (e) {\n  if (TripWire.isTripWire?.(e) || e?.name === 'TripWire') {\n    // e.metadata: { systemTokens, limit } — raise maxTokens or shorten system prompt\n  } else throw e;\n}","preventionTips":["Set maxTokens well above measured system-prompt tokens plus overhead.","Unit-test the processor with your real system prompt.","Keep system prompts short; move static context to retrieval."],"tags":["token-limit","processors","configuration","tripwire"],"backgroundTag":"token-limit-exceeded","analyzedSha":"75dd419e613fe9c39f846ffc500716141b74fda6","analyzedAt":"2026-08-30T00:15:31.844Z","schemaVersion":2},"datasetVersion":"2026-08-30T03:17:51.788Z"}