apify/crawlee · error

Page object was used in HTTP-only request handler

Error message

Page object was used in HTTP-only request handler

What it means

When AdaptivePlaywrightCrawler renders a page via the HTTP-only (cheerio) path, there is no browser Page object. The adapted context exposes a `page` getter that throws this error to prevent developers from accidentally using a nonexistent Playwright Page in an HTTP-only request handler.

Source

Thrown at packages/playwright-crawler/src/internals/adaptive-playwright-crawler.ts:512

                    throw new Error(errorMessage('querySelectorAll'));
                },
                get waitForSelector(): AdaptivePlaywrightCrawlerContext['waitForSelector'] {
                    throw new Error(errorMessage('waitForSelector'));
                },
                get parseWithCheerio(): AdaptivePlaywrightCrawlerContext['parseWithCheerio'] {
                    throw new Error(errorMessage('parseWithCheerio'));
                },
                get enqueueLinks(): AdaptivePlaywrightCrawlerContext['enqueueLinks'] {
                    throw new Error(errorMessage('enqueueLinks'));
                },
            }),
        });
    }

    private async adaptCheerioContext(cheerioContext: CheerioCrawlingContext) {
        return {
            get page(): Page {
                throw new Error('Page object was used in HTTP-only request handler');
            },
            async querySelector(selector: string) {
                return cheerioContext.$(selector).first();
            },
            async querySelectorAll(selector: string) {
                return cheerioContext.$(selector);
            },
            enqueueLinks: async (options: EnqueueLinksOptions = {}) => {
                const urls = extractUrlsFromCheerio(
                    cheerioContext.$,
                    options.selector,
                    options.baseUrl ?? cheerioContext.request.loadedUrl,
                );
                return (await this.enqueueLinks(urls, options, cheerioContext.request)) as unknown as void;
            },
            response: cheerioContext.response,
        };
    }

View on GitHub (pinned to dbe57fb09c)

Solutions

1. Remove or guard usage of context.page so the handler works without a browser Page (use cheerio-based querySelector/querySelectorAll).
2. Force the request through the browser pipeline (e.g., via rendering type hints in request.userData or crawler configuration) whenever page access is genuinely required.
3. Split logic: run page-dependent work in a PlaywrightCrawler and HTTP work in a CheerioCrawler/adaptive handler that never touches page.

Example fix

// before
async requestHandler(context) {
    await context.page.screenshot({ path: 'shot.png' });
}
// after
async requestHandler(context) {
    if (context.page) {
        await context.page.screenshot({ path: 'shot.png' });
    }
}

// before touching page
if (!context.page || isClosedPage(context.page)) skipPageWork();

function hasLivePage(ctx: { page?: unknown }): ctx is { page: Page } {
    const p = ctx.page as Page | undefined;
    return !!p && typeof p.screenshot === 'function' && !p.isClosed();
}

try {
    await context.page.screenshot({ path: 'shot.png' });
} catch (err) {
    if ((err as Error).message === 'Page object was used in HTTP-only request handler') {
        log.info('Skipping screenshot: HTTP-only rendering');
    } else throw err;
}

When it happens

Trigger: Accessing context.page inside a requestHandler when the crawler processed the request over plain HTTP (no browser was launched for that request).

Common situations: Handler written for PlaywrightCrawler reused with AdaptivePlaywrightCrawler; page.screenshot() or page.evaluate() called without checking which rendering path handled the request; intermittent failures because only some requests get browser rendering.

Related errors

• PlaywrightCrawlerOptions.launchContext.proxyUrl is not allow
• saveSnapshot with key ${key} failed. Cause:${(err as Error).
• Function `newContext()` is not available in incognito mode
• Function `newBrowserCDPSession()` is not available in incogn
• Function `startTracing()` is not available in incognito mode

AI-assisted analysis of apify/crawlee@dbe57fb09c (2026-08-30). Data as JSON: /api/errors/8bfe295f0fd54cb8. Report an issue: GitHub.