apify/crawlee · error · Error

This crawler instance is already running, you can add more r

Error message

This crawler instance is already running, you can add more requests to it via `crawler.addRequests()`.

What it means

`crawler.run()` can only execute one crawl per instance at a time; `this.running` is true while a run is in progress. Calling run() again concurrently is rejected, and the message points to `crawler.addRequests()` as the supported way to feed more work into the running crawl.

Source

Thrown at packages/basic-crawler/src/internals/basic-crawler.ts:1700

        const interval = setInterval(log, this.#statusMessageLoggingInterval * 1e3);
        return { log, stop: () => clearInterval(interval) };
    }

    /**
     * Runs the crawler. Returns a promise that resolves once every request has been processed and the crawler's
     * finished-check ({@apilink BasicCrawlerOptions.taskLoopOptions|`taskLoopOptions.isFinishedFunction`}, or the
     * default "the request manager is empty") reports that the crawl is over.
     *
     * We can use the `requests` parameter to enqueue the initial requests — it is a shortcut for
     * running {@apilink BasicCrawler.addRequests|`crawler.addRequests()`} before {@apilink BasicCrawler.run|`crawler.run()`}.
     *
     * @param [requests] The requests to add.
     * @param [options] Options for the request queue.
     */
    async run(requests?: TypedRequestsLike<Routes>, options?: CrawlerRunOptions): Promise<FinalStatistics> {
        if (this.running) {
            throw new Error(
                'This crawler instance is already running, you can add more requests to it via `crawler.addRequests()`.',
            );
        }

        const { purgeRequestQueue, ...addRequestsOptions } = options ?? {};

        if (this.hasFinishedBefore) {
            // When executing the run method for the second time explicitly,
            // we need to purge the RQ to allow processing the same requests again — this is important so users can
            // pass in failed requests back to the `crawler.run()`, otherwise they would be considered as handled and
            // ignored — as a failed request is still handled.
            // `purgeRequestQueue` unset purges only storage the crawler opened itself (see `#purgeableExtent`);
            // `true` also purges a caller-supplied manager, `false` purges nothing.
            if (purgeRequestQueue === undefined && this.#purgeableExtent === 'ambiguous') {
                throw new Error(
                    'Cannot decide what to purge before running again: `sameDomainDelaySecs` paces the request ' +
                        'manager you supplied, so the per-domain queues that have to be emptied are the ' +
                        "crawler's while the manager underneath them is yours. Say which you want: " +

View on GitHub (pinned to dbe57fb09c)

Solutions

Await the current `crawler.run()` promise before calling run() again.
Use `crawler.addRequests(...)` to add URLs to the already-running crawl.
Create a new crawler instance for a concurrent crawl.
Guard with a flag/promise chain so concurrent callers wait for or reuse the active run.

Example fix

// before
setInterval(() => crawler.run(requests), 60_000);
// after
async function loop() {
  while (true) {
    await crawler.run(requests);
    await new Promise((r) => setTimeout(r, 60_000));
  }
}

Defensive patterns

Strategy: type-guard

Validate before calling

if (crawler.running) {
  await crawler.addRequests(newRequests);
} else {
  await crawler.run(newRequests);
}

Type guard

function canRun(c) { return typeof c === 'object' && c !== null && c.running === false; }

Try / catch

try {
  await crawler.run(requests);
} catch (err) {
  if (err.message.includes('already running')) {
    await crawler.addRequests(requests);
  } else throw err;
}

Prevention

Never call run() concurrently on the same crawler instance; serialize runs through a promise chain.
Use addRequests() to feed a running crawl instead of nested run() calls.
Do not call run() from inside request handlers of the same crawler.
Create separate instances when parallel crawls are needed.

When it happens

Trigger: Calling `crawler.run(...)` (or awaiting two run() calls in parallel, or run() from an event handler while a run is active) while a previous `run()` on the same instance has not finished.

Common situations: Scheduling code that fires run() on a timer while the previous crawl is still going; calling run() inside a request handler of the same crawler; awaiting run() in two places by mistake.

Related errors

AI-assisted analysis of apify/crawlee@dbe57fb09c (2026-08-30). Data as JSON: /api/errors/aa45ef1f712fefd1. Report an issue: GitHub.