apify/crawlee · error · Error

Request provider is not initialized!

Error message

Request provider is not initialized!

What it means

Inside the autoscaled pool's runTaskFunction the crawler reads `this.requestManager`; if it is falsy there is no request provider to pull work from, so the task throws. This is an internal invariant check — under normal operation the constructor always initializes a provider before the pool starts.

Source

Thrown at packages/basic-crawler/src/internals/basic-crawler.ts:1249

            this.#maxRequestsPerCrawl = maxRequestsPerCrawl;

            const isMaxPagesExceeded = () =>
                this.#maxRequestsPerCrawl && this.#maxRequestsPerCrawl <= this.handledRequestsCount;

            // eslint-disable-next-line prefer-const
            let { isFinishedFunction, isTaskReadyFunction } = taskLoopOptions;

            // override even if `isFinishedFunction` provided by user - `keepAlive` has higher priority
            if (keepAlive) {
                isFinishedFunction = async () => false;
            }

            const crawlerOwnedTaskLoopConfiguration: Partial<
                Omit<AutoscaledPoolOptions, 'concurrencySystem' | 'consumer'>
            > = {
                runTaskFunction: async () => {
                    const source = this.requestManager;
                    if (!source) throw new Error('Request provider is not initialized!');

                    const request = await this.resolveRequest();
                    if (!request) {
                        return;
                    }

                    // Started here, rather than in `handleRequest`, so that a failure during context pipeline
                    // initialization (e.g. a browser page timing out before the request handler ever runs) is
                    // still accounted for by `failJob` below - which is a no-op without a matching `startJob`.
                    this.statistics.startJob(request.id || request.uniqueKey);

                    const crawlingContext = { request } as { request: Request } & Partial<CrawlingContext>;
                    try {
                        // The transaction spans the whole pipeline call, covering the navigation hooks
                        // and `extendContext` too; `handleRequest` drives its outcome explicitly.
                        await this.runInStorageTransaction(
                            async () =>
                                // Navigation, the navigation hooks and the request handler are timed individually, but the

View on GitHub (pinned to dbe57fb09c)

Solutions

1. Ensure the crawler is constructed and started through the standard `crawler.run()` flow so the request manager is initialized first.
2. If subclassing, call `super()` and let the base constructor set up the request provider before starting any pool.
3. Check that no code sets or clears the internal request manager before the crawl starts.

Example fix

// before (subclass skipping base init)
constructor(opts) { this.startPool(); }
// after
constructor(opts) { super(opts); this.startPool(); }

if (!crawler.requestManager) throw new Error('Crawler has no request provider; call crawler.run() instead of driving internals.');

function hasProvider(c) { return typeof c === 'object' && c !== null && 'requestManager' in c && c.requestManager != null; }

try {
  await crawler.run();
} catch (err) {
  if (err.message.includes('Request provider is not initialized')) {
    // re-create the crawler via its normal constructor and rerun
  } else throw err;
}

When it happens

Trigger: The autoscaled pool's runTaskFunction executing while `this.requestManager` is undefined — e.g. due to abnormal initialization order, subclass overriding initialization, or a pool started before the provider was assigned.

Common situations: Custom subclasses or partial mocking of BasicCrawler that bypass the constructor's provider setup; race conditions when starting the run loop manually; internal state corruption.

Related errors

• fetchNextRequest called on an uninitialized crawler
• OwnedOrInjected value is not initialized yet
• Cannot set() a borrowed OwnedOrInjected value
• OwnedOrInjected value is already initialized
• Recoverable state has not yet been loaded - call initialize(

AI-assisted analysis of apify/crawlee@dbe57fb09c (2026-08-30). Data as JSON: /api/errors/1e6276782d78db91. Report an issue: GitHub.