{"record":{"id":"7442cc2fa3bd3940","repo":"FlowiseAI/Flowise","slug":"crawl-request-failed-crawlresponse-error-un","errorCode":null,"errorMessage":"Crawl request failed: ${crawlResponse.error || 'Unknown error'}","messagePattern":"Crawl request failed: (.+?)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/components/nodes/documentloaders/FireCrawl/FireCrawl.ts","lineNumber":298,"sourceCode":"                }\n            })\n        }\n\n        // Only add scrapeOptions if it has more than just the default values\n        if (Object.keys(scrapeOptions).length > 2) {\n            validParams.scrapeOptions = scrapeOptions\n        }\n\n        try {\n            const parameters = {\n                ...validParams,\n                integration: 'flowise'\n            }\n            const response: AxiosResponse = await this.postRequest(this.apiUrl + '/v1/crawl', parameters, headers)\n            if (response.status === 200) {\n                const crawlResponse = response.data as CrawlResponse\n                if (!crawlResponse.success) {\n                    throw new Error(`Crawl request failed: ${crawlResponse.error || 'Unknown error'}`)\n                }\n\n                if (waitUntilDone) {\n                    return this.monitorJobStatus(crawlResponse.id, headers, pollInterval)\n                } else {\n                    return crawlResponse\n                }\n            } else {\n                this.handleError(response, 'start crawl job')\n            }\n        } catch (error: any) {\n            if (error.response?.data?.error) {\n                throw new Error(`Crawl failed: ${error.response.data.error}`)\n            }\n            throw new Error(`Crawl failed: ${error.message}`)\n        }\n\n        return { success: false, id: '', url: '' }","sourceCodeStart":280,"sourceCodeEnd":316,"githubUrl":"https://github.com/FlowiseAI/Flowise/blob/abe4a8601a058047b350c260676826e21dd14101/packages/components/nodes/documentloaders/FireCrawl/FireCrawl.ts#L280-L316","documentation":"crawlUrl received HTTP 200 from POST /v1/crawl but the body had success:false. The crawl job was not accepted. The error field (or 'Unknown error' if absent) is surfaced. This fires before any job-status polling begins.","triggerScenarios":"URL is invalid or blocked; maxDepth/limit combination rejected; includePaths/excludePaths produced an empty crawl set; account out of credits or rate-limited at job-creation time; URL scheme unsupported (e.g., ftp://).","commonSituations":"Passing a non-http(s) URL; overly restrictive includePaths that exclude the seed; Free-tier quota exceeded; URL blocked upstream by FireCrawl policy.","solutions":["Inspect crawlResponse.error for the exact reason.","Simplify: drop includePaths/excludePaths/maxDepth and retry with a bare URL.","Confirm the account has crawl credits and is not rate-limited (check dashboard).","Ensure the URL uses http or https and is publicly reachable."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"function validateCrawlRequest(url: string, params: any): void {\n  try { new URL(url) } catch { throw new Error(`crawlUrl url is invalid: ${url}`) }\n  if (!/^https?:/i.test(new URL(url).protocol)) throw new Error('crawlUrl requires http(s) url')\n  if (params?.maxDepth !== undefined && (!Number.isFinite(params.maxDepth) || params.maxDepth < 0)) {\n    throw new Error('crawlUrl maxDepth must be a non-negative number')\n  }\n  if (params?.limit !== undefined && (!Number.isFinite(params.limit) || params.limit <= 0)) {\n    throw new Error('crawlUrl limit must be positive')\n  }\n}\n// validateCrawlRequest(url, params) before app.crawlUrl(...)","typeGuard":null,"tryCatchPattern":"try {\n  await app.crawlUrl(url, params, true, pollInterval, idempotencyKey)\n} catch (error) {\n  const msg = error instanceof Error ? error.message : String(error)\n  if (/Crawl request failed:/.test(msg)) {\n    // API rejected the crawl - simplify params and retry once\n    await app.crawlUrl(url, null, true, pollInterval)\n  } else throw error\n}","preventionTips":["Start with a bare URL and no params to confirm crawl acceptance, then add constraints.","Ensure the account has crawl credits and the URL is publicly reachable.","Use an http(s) URL; other schemes are rejected."],"tags":["firecrawl","api-errors","crawl"],"backgroundTag":null,"analyzedSha":"abe4a8601a058047b350c260676826e21dd14101","analyzedAt":"2026-08-12T16:04:40.823Z","schemaVersion":2},"datasetVersion":"2026-08-12T18:17:37.767Z"}