{"record":{"id":"3719bca36b0bcde5","repo":"apify/crawlee","slug":"dynamic-message-extracted-from-proxied-error-via","errorCode":null,"errorMessage":"<dynamic: message extracted from proxied error via getMessageFromError>","messagePattern":"<dynamic: message extracted from proxied error via getMessageFromError>","errorType":"exception","errorClass":"SessionError","httpStatus":null,"severity":"warning","filePath":"packages/http-crawler/src/internals/http-crawler.ts","lineNumber":708,"sourceCode":"\n    /**\n     * Function to make the HTTP request. It performs optimizations\n     * on the request such as only downloading the request body if the\n     * received content type matches text/html, application/xml, application/xhtml+xml.\n     */\n    private async requestFunction({ request, session, proxyUrl }: RequestFunctionOptions): Promise<Response> {\n        const opts = this.getRequestOptions(request, session, proxyUrl);\n\n        try {\n            return await this.requestAsBrowser(opts, session);\n        } catch (e) {\n            if (e instanceof Error && e.constructor.name === 'TimeoutError') {\n                this.handleRequestTimeout(session);\n                return new Response(); // this will never happen, as handleRequestTimeout always throws\n            }\n\n            if (this.isProxyError(e as Error)) {\n                throw new SessionError(this.getMessageFromError(e as Error) as string);\n            } else {\n                throw e;\n            }\n        }\n    }\n\n    /**\n     * Encodes and parses response according to the provided content type\n     */\n    private async parseResponse(request: CrawleeRequest, response: Response) {\n        const { status } = response;\n        const { type, charset } = parseContentTypeFromResponse(response);\n        const { response: reencodedResponse, encoding } = this.encodeResponse(request, response, charset);\n        const contentType = { type, encoding };\n\n        if (status >= 400 && status <= 599) {\n            this.statistics.registerStatusCode(status);\n        }","sourceCodeStart":690,"sourceCodeEnd":726,"githubUrl":"https://github.com/apify/crawlee/blob/dbe57fb09ca607ad59dcf998f3925ef9ac3bb26c/packages/http-crawler/src/internals/http-crawler.ts#L690-L726","documentation":"Inside the crawler's requestFunction, errors are inspected after the request fails; if `isProxyError()` classifies the thrown error as a proxy-level failure (e.g. proxy connection refused, 407, tunnel errors), it is re-thrown as a SessionError carrying the original message extracted via getMessageFromError. This marks the session as bad so the request is retried through a different proxy instead of failing the whole request.","triggerScenarios":"The underlying HTTP request (got) rejects with a proxy-related error while a proxyConfiguration is in use — e.g. proxy authentication failed, proxy unreachable, or the proxy returned an error response.","commonSituations":"Expired or wrong proxy credentials; datacenter IPs blocked by the target; proxy provider outage; residential proxy session exhausted; running locally without the proxy reachable from the network.","solutions":["Verify proxy credentials (username/password) and proxy URL in your ProxyConfiguration are current.","Test the proxy directly (curl -x proxy_url target_url) to confirm it works outside crawlee.","Ensure session pool is enabled so bad sessions are retired and the request retries on a healthy proxy.","Check your proxy provider's dashboard for quota exhaustion or IP pool health; switch pools if needed."],"exampleFix":"// before\nproxyConfiguration: await Actor.createProxyConfiguration({ groups: ['RESIDENTIAL'] }),\n\n// after: validate proxy before crawling\nconst proxyConfiguration = await Actor.createProxyConfiguration({ groups: ['RESIDENTIAL'] });\nconst proxyUrl = await proxyConfiguration.newUrl();\nconst res = await got('https://api.apify.com/v2/public-proxy-ip', { proxyUrl, throwHttpErrors: false });\nif (res.statusCode >= 400) throw new Error(`Proxy unhealthy: ${proxyUrl}`);","handlingStrategy":"try-catch","validationCode":"const proxyUrl = await proxyConfiguration.newUrl();\nconst probe = await got('https://httpbin.org/ip', { proxyUrl, throwHttpErrors: false, timeout: 10000 });\nif (probe.statusCode !== 200) throw new Error(`Proxy probe failed: ${probe.statusCode}`);","typeGuard":"function isProxySessionError(err: unknown): err is SessionError {\n  return err instanceof SessionError && /proxy/i.test(err.message);\n}","tryCatchPattern":"try {\n  await crawler.run(requests);\n} catch (err) {\n  if (err instanceof SessionError && /proxy|tunnel|407/i.test(err.message)) {\n    log.error(`Proxy infrastructure problem, aborting: ${err.message}`);\n    return;\n  }\n  throw err;\n}","preventionTips":["Validate proxy credentials and quota before starting large crawls.","Smoke-test one request through the proxy pipeline before full runs.","Prefer provider proxy groups with health monitoring; rotate groups on repeated proxy errors.","Log proxy session IDs on failure to correlate with provider logs."],"tags":["proxy","network","session","retry"],"backgroundTag":"proxy-connection-failed","analyzedSha":"dbe57fb09ca607ad59dcf998f3925ef9ac3bb26c","analyzedAt":"2026-08-30T22:22:28.328Z","schemaVersion":2},"datasetVersion":"2026-08-30T23:17:21.991Z"}