{"record":{"id":"846c5f12ce22c1aa","repo":"FlowiseAI/Flowise","slug":"crawl-failed-error-message","errorCode":null,"errorMessage":"Crawl failed: ${error.message}","messagePattern":"Crawl failed: (.+?)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/components/nodes/documentloaders/FireCrawl/FireCrawl.ts","lineNumber":313,"sourceCode":"            if (response.status === 200) {\n                const crawlResponse = response.data as CrawlResponse\n                if (!crawlResponse.success) {\n                    throw new Error(`Crawl request failed: ${crawlResponse.error || 'Unknown error'}`)\n                }\n\n                if (waitUntilDone) {\n                    return this.monitorJobStatus(crawlResponse.id, headers, pollInterval)\n                } else {\n                    return crawlResponse\n                }\n            } else {\n                this.handleError(response, 'start crawl job')\n            }\n        } catch (error: any) {\n            if (error.response?.data?.error) {\n                throw new Error(`Crawl failed: ${error.response.data.error}`)\n            }\n            throw new Error(`Crawl failed: ${error.message}`)\n        }\n\n        return { success: false, id: '', url: '' }\n    }\n\n    async extract(\n        request: ExtractRequest,\n        waitUntilDone: boolean = true,\n        pollInterval: number = 2\n    ): Promise<ExtractResponse | ExtractStatusResponse> {\n        const headers = this.prepareHeaders()\n\n        // Create a clean payload with only valid parameters\n        const validParams: any = {\n            urls: request.urls\n        }\n\n        // Add optional parameters if they exist and are not empty","sourceCodeStart":295,"sourceCodeEnd":331,"githubUrl":"https://github.com/FlowiseAI/Flowise/blob/abe4a8601a058047b350c260676826e21dd14101/packages/components/nodes/documentloaders/FireCrawl/FireCrawl.ts#L295-L331","documentation":"Fallback branch in crawlUrl's catch: the error had no error.response.data.error, so only error.message survives. Covers pure transport errors (ECONNRESET, ETIMEDOUT, ENOTFOUND), the 'Crawl request failed' throw from [117], the 'Too many redirects' or SSRF deny-list throws from secureAxiosRequest, and the non-200 handleError throws when response.data had no error field.","triggerScenarios":"Network cannot reach api.firecrawl.dev; TLS handshake failed; too many redirects; URL/IP blocked by the SSRF guard in secureAxiosRequest; handleError threw 'Unexpected error ... Status code: <n>' for a status not in [402,408,409,500].","commonSituations":"DNS outage; corporate firewall blocking outbound HTTPS; self-hosted FireCrawl apiUrl pointing at an internal IP that the SSRF guard blocks; intermittent 502/503 from a load balancer (not in handleError's list).","solutions":["Read the message: 'Too many redirects' or a URL-validation message points at the SSRF guard; ETIMEDOUT/ECONNRESET points at transport; 'Crawl request failed' points at [117].","Confirm outbound HTTPS to api.firecrawl.dev works from the host (curl -v).","If using a self-hosted apiUrl on internal infrastructure, ensure the URL is allowed by resolveAndValidate or is publicly resolvable.","For transient transport errors, retry with backoff."],"exampleFix":"// before\n} catch (error: any) {\n  if (error.response?.data?.error) {\n    throw new Error(`Crawl failed: ${error.response.data.error}`)\n  }\n  throw new Error(`Crawl failed: ${error.message}`)\n}\n// after - preserve cause for diagnosis\n} catch (error: any) {\n  const detail = error.response?.data?.error ?? error.message\n  throw new Error(`Crawl failed: ${detail}`, { cause: error })\n}","handlingStrategy":"retry","validationCode":"import { resolve as dnsResolve } from 'dns/promises'\n\nasync function assertFirecrawlDns(hostname = 'api.firecrawl.dev'): Promise<void> {\n  try { await dnsResolve(hostname) }\n  catch { throw new Error(`DNS lookup failed for ${hostname} - cannot reach FireCrawl`) }\n}\n// await assertFirecrawlDns(new URL(apiUrl).hostname) before crawlUrl","typeGuard":null,"tryCatchPattern":"async function crawlWithRetry(app: FirecrawlApp, url: string, params: any, attempts = 3) {\n  for (let i = 0; i < attempts; i++) {\n    try {\n      return await app.crawlUrl(url, params)\n    } catch (error) {\n      const msg = error instanceof Error ? error.message : String(error)\n      if (/ETIMEDOUT|ECONNRESET|ENOTFOUND|EAI_AGAIN/.test(msg) && i < attempts - 1) {\n        await new Promise((r) => setTimeout(r, 2 ** i * 500))\n        continue\n      }\n      throw error\n    }\n  }\n  throw new Error('unreachable')\n}","preventionTips":["Confirm outbound HTTPS to api.firecrawl.dev works from the host (curl -v).","For self-hosted apiUrl on internal hosts, ensure the SSRF guard (resolveAndValidate) allows the IP.","Retry transient transport errors with exponential backoff; do not retry SSRF/redirect errors."],"tags":["firecrawl","network","crawl","ssrf","error-handling"],"backgroundTag":null,"analyzedSha":"abe4a8601a058047b350c260676826e21dd14101","analyzedAt":"2026-08-12T16:04:40.823Z","schemaVersion":2},"datasetVersion":"2026-08-12T18:17:37.767Z"}