apify/crawlee · error · Error

The HTTP method CONNECT is not supported by the GotScrapingH

Error message

The HTTP method CONNECT is not supported by the GotScrapingHttpClient.

What it means

GotScrapingHttpClient does not implement the HTTP CONNECT method (used for tunneling, e.g. HTTPS proxies). validateRequest fails for CONNECT requests, so fetch() throws before any network call is made.

Source

Thrown at packages/got-scraping-client/src/index.ts:41

        for (const [key, value] of Object.entries(headers)) {
            if (key.startsWith(':') || value === undefined) continue;
            if (Array.isArray(value)) {
                for (const v of value) yield [key, v];
            } else {
                yield [key, value];
            }
        }
    }

    private parseHeaders(headers: Record<string, string | string[] | undefined>): Headers {
        return new Headers([...this.iterateHeaders(headers)]);
    }

    override async fetch(request: Request, options?: RequestInit & CustomFetchOptions): Promise<Response> {
        const { proxyUrl, redirect, ignoreTlsErrors } = options ?? {};

        if (!this.validateRequest(request)) {
            throw new Error(`The HTTP method CONNECT is not supported by the GotScrapingHttpClient.`);
        }

        const gotResult = await gotScraping({
            url: request.url!,
            method: request.method as Options['method'],
            headers: Object.fromEntries(request.headers.entries()),
            body: request.body ? Readable.fromWeb(request.body as any) : undefined,
            proxyUrl,
            signal: options?.signal ?? undefined,
            followRedirect: redirect === 'follow',
            ...(ignoreTlsErrors ? { https: { rejectUnauthorized: false } } : {}),
        });

        const responseHeaders = this.parseHeaders(gotResult.headers);

        return new ResponseWithUrl(new Uint8Array(gotResult.rawBody), {
            headers: responseHeaders,
            status: gotResult.statusCode,

View on GitHub (pinned to dbe57fb09c)

Solutions

1. Use a different HTTP client that supports CONNECT for tunneling use cases
2. Restrict your routing layer to standard methods (GET/POST/PUT/DELETE/PATCH/HEAD/OPTIONS)
3. Handle CONNECT requests separately (e.g. via node's http/https tunneling) before they reach this client
4. Map CONNECT requests to a supported flow instead of proxying raw tunnels

Example fix

// before
const res = await gotClient.fetch(new Request(url, { method: 'CONNECT' })); // throws
// after
if (request.method === 'CONNECT') {
  return handleConnectTunnel(request); // separate tunneling implementation
}
const res = await gotClient.fetch(request);

if (request.method === 'CONNECT') {
  throw new Error('CONNECT is not supported by GotScrapingHttpClient; use a tunneling client');
}

const isSupportedMethod = (m) => ['GET','POST','PUT','DELETE','PATCH','HEAD','OPTIONS'].includes(m?.toUpperCase());

try {
  return await gotClient.fetch(request);
} catch (err) {
  if (/CONNECT is not supported/.test(String(err))) {
    return handleWithTunnelingClient(request);
  }
  throw err;
}

When it happens

Trigger: Calling httpClient.fetch(new Request(url, { method: 'CONNECT' })) or route() with a request whose method is CONNECT against the GotScraping client.

Common situations: Building a generic proxy server on top of the HTTP client; forwarding arbitrary user requests (browsers issue CONNECT for HTTPS through proxies); generated code that passes methods through unfiltered.

Related errors

• Could not parse cookie header string: ${cookieHeaderString}
• Request with GET method cannot have a payload.
• Cannot fetch a request list from ${requestsFromUrl}: ${err}
• Cannot fetch a request list from ${requestsFromUrl}: ${err}
• Too many redirects (${maxRedirects}) while requesting ${curr

AI-assisted analysis of apify/crawlee@dbe57fb09c (2026-08-30). Data as JSON: /api/errors/a685f8304f4c63ab. Report an issue: GitHub.