apify/crawlee · critical · CriticalError

The ConcurrencySystem this AutoscaledPool borrows has not be

Error message

The ConcurrencySystem this AutoscaledPool borrows has not been started, so system load would not be monitored and the concurrency would never be adjusted. Whoever creates a ConcurrencySystem owns its lifecycle: call `await concurrencySystem.start()` before running the pools or crawlers that use it, and `await concurrencySystem.stop()` once they are all done.

What it means

AutoscaledPool borrows a ConcurrencySystem owned by the caller. If `run()` is called while that system isn't running, load would never be monitored and concurrency never adjusted, so Crawlee throws a CriticalError up front (on the awaited path) instead of silently hanging.

Source

Thrown at packages/core/src/autoscaling/autoscaled_pool.ts:242

     * The number of parallel tasks currently booked against the governor. When it is shared, this counts every
     * borrowing pool's tasks, not just this one's.
     */
    get currentConcurrency(): number {
        return this.#concurrencySystem.currentConcurrency;
    }

    /**
     * Runs the auto-scaled pool. Returns a promise that gets resolved or rejected once
     * all the tasks are finished or one of them fails.
     *
     * Throws if the {@apilink IConcurrencySystem|concurrency system} it borrows was never started — the pool assumes
     * a running governor and cannot start one it does not own.
     */
    async run(): Promise<void> {
        // Checked here, on an awaited path — the capacity queries inside the task loop run from intervals and
        // `setImmediate`, where a throw would become an unhandled rejection and hang `run()` forever.
        if (!this.#concurrencySystem.isRunning) {
            throw new CriticalError(
                'The ConcurrencySystem this AutoscaledPool borrows has not been started, so system load would not be ' +
                    'monitored and the concurrency would never be adjusted. Whoever creates a ConcurrencySystem owns ' +
                    'its lifecycle: call `await concurrencySystem.start()` before running the pools or crawlers that ' +
                    'use it, and `await concurrencySystem.stop()` once they are all done.',
            );
        }

        const poolPromise = new Promise((resolve, reject) => {
            this.#resolve = resolve;
            this.#reject = reject;
        });

        // This is here because if we scale down to let's say 1, then after each promise is finished
        // this.maybeRunTask() doesn't trigger another one. So if that 1 instance gets stuck it results
        // in the crawler getting stuck and even after scaling up it never triggers another promise.
        this.#maybeRunInterval = betterSetInterval(this.maybeRunTask, this.#maybeRunIntervalMillis);

        try {

View on GitHub (pinned to dbe57fb09c)

Solutions

Call `await concurrencySystem.start()` before `await pool.run()`.
Keep the system running until every pool/crawler using it has finished; only then `await concurrencySystem.stop()`.
Prefer high-level Crawler APIs (which own the system lifecycle) over hand-rolled AutoscaledPool wiring.

Example fix

// before
const pool = new AutoscaledPool({ concurrencySystem: system, ... });
await pool.run();
// after
await system.start();
const pool = new AutoscaledPool({ concurrencySystem: system, ... });
await pool.run();
await system.stop();

Defensive patterns

Strategy: validation

Validate before calling

if (!concurrencySystem.isRunning) { throw new Error('start the ConcurrencySystem before running pools'); }
await pool.run();

Try / catch

try { await pool.run(); } catch (err) { if (err instanceof CriticalError && err.message.includes('ConcurrencySystem')) { await system.start(); await pool.run(); } else { throw err; } }

Prevention

Always pair `system.start()` before pools and `system.stop()` after all complete
Prefer high-level Crawler classes that own the ConcurrencySystem lifecycle
In tests, assert `system.isRunning` before invoking run()

When it happens

Trigger: Manually constructing a ConcurrencySystem (or a low-level AutoscaledPool) and calling `pool.run()` without first `await concurrencySystem.start()`; or calling `stop()` on the system before the pool finishes.

Common situations: Custom orchestration code composing AutoscaledPool/Crawler instances directly instead of through default factories; stopping the shared concurrency system while a second pool still runs.

Related errors

AI-assisted analysis of apify/crawlee@dbe57fb09c (2026-08-30). Data as JSON: /api/errors/4114e7209db8134e. Report an issue: GitHub.