What causes "${response.error || 'LLMs.txt generation failed'}" in mastra-ai/mastra?

Log/inspect response.error for the concrete API cause (auth vs rate limit vs job failure).

mastra-ai/mastra · error · Error

${response.error || 'LLMs.txt generation failed'}

Error message

${response.error || 'LLMs.txt generation failed'}

What it means

The LLMs.txt generation call's response is checked for response.success after withRetry; on failure the server throws the API's error message or 'LLMs.txt generation failed'. LLMs.txt generation is an async Firecrawl job (crawl + summarize), so the error can come from job state as well as auth/limits.

Source

Thrown at packages/mcp/src/__fixtures__/fire-crawl-complex-schema.ts:912

      case 'firecrawl_generate_llmstxt': {
        if (!isGenerateLLMsTextOptions(args)) {
          throw new Error('Invalid arguments for firecrawl_generate_llmstxt');
        }

        try {
          const { url, ...params } = args;
          const generateStartTime = Date.now();

          safeLog('info', `Starting LLMs.txt generation for URL: ${url}`);

          const response = await withRetry(
            async () => client.generateLLMsText(url, { ...params }),
            'LLMs.txt generation',
          );

          if (!response.success) {
            throw new Error(response.error || 'LLMs.txt generation failed');
          }

          safeLog('info', `LLMs.txt generation completed in ${Date.now() - generateStartTime}ms`);

          let resultText = '';

          if ('data' in response) {
            resultText = `LLMs.txt content:\n\n${response.data.llmstxt}`;

            if (args.showFullText && response.data.llmsfulltxt) {
              resultText += `\n\nLLMs-full.txt content:\n\n${response.data.llmsfulltxt}`;
            }
          }

          return {
            content: [{ type: 'text', text: trimResponseText(resultText) }],
            isError: false,
          };

View on GitHub (pinned to 75dd419e61)

Solutions

Log/inspect response.error for the concrete API cause (auth vs rate limit vs job failure).
Verify API key and credit balance; large sites consume many credits.
Lower maxUrls or retry with backoff for transient failures; re-run the generation.
Check Firecrawl status page for degraded crawl/search performance before retrying.

Example fix

// before
if (!response.success) {
  throw new Error(response.error || 'LLMs.txt generation failed');
}
// after
if (!response.success) {
  const detail = response.error || 'no detail';
  if (/timeout|expired/i.test(detail)) throw new Error(`Generation job timed out, retry with smaller maxUrls: ${detail}`);
  throw new Error(`LLMs.txt generation failed: ${detail}`);
}

Defensive patterns

Strategy: retry

Validate before calling

// Pre-flight: valid URL + key present; scope large sites down
if (!process.env.FIRECRAWL_API_KEY) throw new Error('FIRECRAWL_API_KEY missing');
if (!/^https?:\/\//.test(args?.url ?? '')) throw new TypeError('url must be absolute http(s)');
if ((args.maxUrls ?? 0) > 500) console.warn('maxUrls>500 may exhaust credits or time out');

Type guard

function isLlmsTxtSuccess(res: unknown): res is { success: true; data: { llmstxt?: string; llmsFullTxt?: string } } {
  return typeof res === 'object' && res !== null && (res as any).success === true;
}

Try / catch

try {
  const res = await generateLlmsTxt(url, params);
  if (!res.success) throw new Error(res.error || 'LLMs.txt generation failed');
} catch (e) {
  const msg = e instanceof Error ? e.message : String(e);
  if (/429|rate|timeout|temporarily/i.test(msg)) return withBackoff(() => generateLlmsTxt(url, { ...params, maxUrls: Math.min(params.maxUrls ?? 100, 50) }), 3);
  if (/401|403|credit/i.test(msg)) throw new Error('Fix Firecrawl auth/credits: ' + msg);
  throw e;
}

Prevention

Cap maxUrls for large sites to avoid timeouts and credit drain.
Retry transient failures with backoff; do not blindly retry credit errors.
Verify API key before long-running generation jobs.
Log the raw response.error to distinguish job failures from auth failures.

When it happens

Trigger: response.success is false after generation completes/fails — invalid API key, rate limits during the underlying crawl, credit exhaustion, the target site blocking the crawl, or the generation job expiring.

Common situations: Generating LLMs.txt for very large sites where the job times out or burns credits; sites with bot protection returning failures for every crawled page; intermittent Firecrawl 5xx during long jobs.

Related errors

AI-assisted analysis of mastra-ai/mastra@75dd419e61 (2026-08-30). Data as JSON: /api/errors/5c6e56b980931cc7. Report an issue: GitHub.