{"record":{"id":"f3de9b68c575f350","repo":"apify/crawlee","slug":"error-possible-values-cloudflare-challenge-f","errorCode":null,"errorMessage":"${error} (possible values: 'Cloudflare challenge failed, found selectors: ${...}', 'Found selectors: ${...}', 'Received blocked status code: ${statusCode}')","messagePattern":"(.+?) \\(possible values: 'Cloudflare challenge failed, found selectors: (.+?)', 'Found selectors: (.+?)', 'Received blocked status code: (.+?)'\\)","errorType":"exception","errorClass":"SessionError","httpStatus":null,"severity":"warning","filePath":"packages/browser-crawler/src/internals/browser-crawler.ts","lineNumber":769,"sourceCode":"     */\n    protected override async runRequestHandler(crawlingContext: ExtendedContext): Promise<void> {\n        try {\n            await super.runRequestHandler(crawlingContext);\n        } finally {\n            if (!crawlingContext.request.skipNavigation) {\n                try {\n                    await this.persistCookiesFromPage(crawlingContext);\n                } catch {\n                    // Page may already be closed on some failure paths; ignore.\n                }\n            }\n        }\n    }\n\n    private async handleBlockedRequestByContent(crawlingContext: BrowserCrawlingContext<Page, Response>) {\n        if (this.retryOnBlocked) {\n            const error = await this.isRequestBlocked(crawlingContext);\n            if (error) throw new SessionError(error);\n        }\n\n        return {};\n    }\n\n    private async restoreRequestState(crawlingContext: CrawlingContext) {\n        crawlingContext.request.state = RequestState.REQUEST_HANDLER;\n        return {};\n    }\n\n    private async applyCookies(\n        { session, request, page }: BrowserCrawlingContext<Page, Response>,\n        preHooksCookies: string,\n        postHooksCookies: string,\n    ) {\n        const sessionCookie = session\n            ? (await session.cookieJar.getCookies(request.url)).map(toughCookieToBrowserPoolCookie)\n            : [];","sourceCodeStart":751,"sourceCodeEnd":787,"githubUrl":"https://github.com/apify/crawlee/blob/dbe57fb09ca607ad59dcf998f3925ef9ac3bb26c/packages/browser-crawler/src/internals/browser-crawler.ts#L751-L787","documentation":"`handleBlockedRequestByContent()` asks `isRequestBlocked()` whether the loaded page looks like an anti-bot block (Cloudflare challenge markers, known block selectors, or a blocked HTTP status code). When it returns a description and `retryOnBlocked` is enabled, the crawler throws a `SessionError` carrying that description so the session is retired and the request retried with a fresh session.","triggerScenarios":"`retryOnBlocked: true` (or default) and the loaded page content matches block heuristics: Cloudflare challenge selectors present, known captcha/block selectors found in the DOM, or the response status code is on the blocked list (e.g. 403/429) for a target behind Cloudflare, PerimeterX, or similar protection.","commonSituations":"Scraping Cloudflare-protected sites with a headless browser that fails the challenge; a datacenter proxy IP range that is widely blocked; sessions whose cookies/user-agent fingerprint got flagged; after the target site upgrades its bot protection and old selectors start matching.","solutions":["Let the crawler retry automatically with a fresh session (the `SessionError` marks the session bad) and increase `maxRequestRetries` if blocks are transient","Improve the setup: use higher-quality residential proxies (`proxyConfiguration`), keep `useSessionPool` enabled, and set realistic browser fingerprints","Lower request concurrency and add delays so the domain is not hammered (rate-limit-triggered blocks)","Check `isRequestBlocked`/`blockSelector` configuration if you customized blocked-status codes — your site may be returning an unexpected status that now counts as blocked"],"exampleFix":"// before\nconst crawler = new PlaywrightCrawler({ /* default settings, no proxies */ });\n// after\nconst crawler = new PlaywrightCrawler({\n    retryOnBlocked: true,\n    useSessionPool: true,\n    maxConcurrency: 5,\n    proxyConfiguration: await ProxyConfiguration.create({\n        proxyUrls: ['http://residential-proxy:8000'],\n    }),\n});","handlingStrategy":"retry","validationCode":"// Detect the block yourself before/alongside the crawler so you can react (rotate session/proxy):\nconst html = await page.content();\nconst blocked = await crawler.isRequestBlocked?.({ page, response, request }) ??\n    /Just a moment|Attention Required/i.test(html);","typeGuard":"function isSessionError(e: unknown): e is SessionError {\n    return e instanceof SessionError || (e instanceof Error && /blocked|Cloudflare|blocked status code/i.test(e.message));\n}","tryCatchPattern":"crawler.failedRequestHandler = async ({ request, error }) => {\n    if (error instanceof SessionError && /blocked/i.test(error.message)) {\n        log.warning(`Blocked after all retries: ${request.url} — ${error.message}`);\n        // persist URL for a later rerun with fresh proxies\n    }\n};","preventionTips":["Always run with a session pool and retryOnBlocked enabled for protected sites","Use residential/mobile proxies and realistic fingerprints instead of datacenter IPs","Throttle concurrency and add randomized delays against protected domains"],"tags":["anti-bot","session","cloudflare","blocked-request"],"backgroundTag":"request-blocked-by-antibot","analyzedSha":"dbe57fb09ca607ad59dcf998f3925ef9ac3bb26c","analyzedAt":"2026-08-30T22:22:28.328Z","schemaVersion":2},"datasetVersion":"2026-08-30T23:17:21.991Z"}