FlowiseAI/Flowise · error · Error

Firecrawl: Failed to scrape URL. Error: ${response.error}

Error message

Firecrawl: Failed to scrape URL. Error: ${response.error}

What it means

Thrown by FireCrawlLoader.load() in scrape mode when app.scrapeUrl resolves but `success === false`. Embeds `response.error` (the FireCrawl-supplied reason). Duplicates the throw inside scrapeUrl itself, so reaching this line means scrapeUrl stopped throwing.

Source

Thrown at packages/components/nodes/documentloaders/FireCrawl/FireCrawl.ts:603

                throw new Error(`Firecrawl: Failed to search. Warning: ${response.warning}`)
            }

            // Convert search results to FirecrawlDocument format
            firecrawlDocs = (response.data || []).map((result) => ({
                markdown: result.description,
                metadata: {
                    title: result.title,
                    sourceURL: result.url,
                    description: result.description
                }
            }))
        } else if (this.mode === 'scrape') {
            if (!this.url) {
                throw new Error('Firecrawl: URL is required for scrape mode')
            }
            const response = await app.scrapeUrl(this.url, this.params)
            if (!response.success) {
                throw new Error(`Firecrawl: Failed to scrape URL. Error: ${response.error}`)
            }
            firecrawlDocs = [response.data as FirecrawlDocument]
        } else if (this.mode === 'crawl') {
            if (!this.url) {
                throw new Error('Firecrawl: URL is required for crawl mode')
            }
            const response = await app.crawlUrl(this.url, this.params)
            if ('status' in response) {
                if (response.status === 'failed') {
                    throw new Error('Firecrawl: Crawl job failed')
                }
                firecrawlDocs = response.data || []
            } else {
                if (!response.success) {
                    throw new Error(`Firecrawl: Failed to scrape URL. Error: ${response.error}`)
                }
                firecrawlDocs = [response.data as FirecrawlDocument]
            }

View on GitHub (pinned to abe4a8601a)

Solutions

1. Read `response.error` to get the upstream reason and act on it.
2. Retry once — many scrape failures are transient (rate limit, temporary block).
3. Try a simpler page or switch to crawl mode to spread load.
4. If mocking, return `{ success: true, data: {...} }`.

Example fix

// before
const response = await app.scrapeUrl(this.url, this.params)
if (!response.success) throw new Error(`Firecrawl: Failed to scrape URL. Error: ${response.error}`)

// after
const response = await app.scrapeUrl(this.url, this.params)
if (!response.success) throw new Error(`Firecrawl scrape of ${this.url} failed: ${response.error}`)

try { new URL(url) } catch { throw new Error(`invalid url: ${url}`) }

function isScrapeSuccess(r: ScrapeResponse | undefined): r is ScrapeResponse & { success: true } {
  return !!r && r.success === true
}

for (let i = 0; i < 2; i++) {
  try { return await loader.load() }
  catch (e) { if (/Failed to scrape URL/.test((e as Error).message) && i === 0) continue; throw e }
}

When it happens

Trigger: scrapeUrl returns `{ success: false, error: '...' }` without throwing. Possible if the library is patched, or in tests with a mock returning success:false. Common upstream reasons: invalid URL, blocked site, timeout.

Common situations: Target URL returns 4xx/5xx; site blocks FireCrawl user-agent; non-HTML resource; mock test double returning failure shape.

Related errors

• Firecrawl: Failed to search. Warning: ${response.warning}
• Firecrawl: URL is required for scrape mode
• Search request failed: ${searchResponse.warning || 'Unknown
• Firecrawl API key not set. You can set it as FIRECRAWL_API_K
• Firecrawl: Query is required for search mode

AI-assisted analysis of FlowiseAI/Flowise@abe4a8601a (2026-08-12). Data as JSON: /api/errors/ae357352ad516db3. Report an issue: GitHub.