apify/crawlee · error · Error

availableMemoryRatio is not set in configuration.

Error message

availableMemoryRatio is not set in configuration.

What it means

When starting the memory load signal and no absolute `memoryMbytes` limit is given, Crawlee falls back to `availableMemoryRatio` from configuration. If neither is set (ratio is 0/undefined) it cannot compute a memory ceiling and throws during `start()`.

Source

Thrown at packages/core/src/autoscaling/memory_load_signal.ts:88

    async start(context: LoadSignalStartContext): Promise<void> {
        this.#store.useSampleWindow(context.maxSampleWindowMillis);
        // A new session starts from a clean slate, so it is not judged on measurements from before the downtime.
        this.#store.clear();

        // Resolved here rather than in the constructor: an instance built ahead of time (to be wrapped, or shared
        // between systems) must not capture whichever services happened to be registered at that moment.
        this.#config = serviceLocator.getConfiguration();
        this.#events = serviceLocator.getEventManager();
        this.#log = serviceLocator.getLogger().child({ prefix: 'MemoryLoadSignal' });

        const memoryMbytes = this.#config.memoryMbytes ?? 0;

        if (memoryMbytes > 0) {
            this.#maxMemoryBytes = memoryMbytes * 1024 * 1024;
        } else {
            this.#maxMemoryRatio = this.#config.availableMemoryRatio;
            if (!this.#maxMemoryRatio) {
                throw new Error('availableMemoryRatio is not set in configuration.');
            } else {
                this.#log.debug(
                    `Setting max memory of this run to ${this.#maxMemoryRatio * 100} % of available memory. ` +
                        'Use the CRAWLEE_MEMORY_MBYTES or CRAWLEE_AVAILABLE_MEMORY_RATIO environment variable to override it.',
                );
            }
            // Fallback memory measurement in case memTotalBytes is missing from SystemInfo.
            this.#maxMemoryBytes = await this.getTotalMemoryBytes();
        }

        this.#events.on(EventType.SYSTEM_INFO, this.handle);
    }

    async stop(): Promise<void> {
        this.#events?.off(EventType.SYSTEM_INFO, this.handle);
        this.#events = undefined;
    }

View on GitHub (pinned to dbe57fb09c)

Solutions

Set `availableMemoryRatio` in the Configuration (e.g. 0.5) passed to the autoscaling setup.
Or set an absolute limit via `memoryMbytes` / the CRAWLEE_MEMORY_MBYTES env var.
Or set CRAWLEE_AVAILABLE_MEMORY_RATIO in the environment.

Example fix

// before
const config = new Configuration();
const signal = new MemoryLoadSignal({ config });
await signal.start();
// after
const config = new Configuration({ availableMemoryRatio: 0.5 });
const signal = new MemoryLoadSignal({ config });
await signal.start();

Defensive patterns

Strategy: validation

Validate before calling

const ratio = config.get('availableMemoryRatio');
const mbytes = Number(process.env.CRAWLEE_MEMORY_MBYTES ?? 0);
if (!ratio && mbytes <= 0) throw new Error('Set CRAWLEE_MEMORY_MBYTES or CRAWLEE_AVAILABLE_MEMORY_RATIO');

Type guard

function hasMemoryLimit(cfg: Configuration): boolean { return Boolean(cfg.get('availableMemoryRatio')) || Number(process.env.CRAWLEE_MEMORY_MBYTES) > 0; }

Try / catch

try { await signal.start(); } catch (err) { if (err.message.includes('availableMemoryRatio')) { /* reconfigure with a ratio and retry */ } else { throw err; } }

Prevention

Set CRAWLEE_MEMORY_MBYTES or CRAWLEE_AVAILABLE_MEMORY_RATIO in the environment
When building Configuration programmatically, include availableMemoryRatio explicitly
Smoke-test autoscaling startup in CI to catch config drift

When it happens

Trigger: Constructing MemoryLoadSignal / starting autoscaling with no `memoryMbytes` and a Configuration whose `availableMemoryRatio` is unset (and env vars CRAWLEE_MEMORY_MBYTES / CRAWLEE_AVAILABLE_MEMORY_RATIO unset).

Common situations: Programmatic Configuration objects that drop defaults; tests constructing Configuration from scratch; environments where config was built without the default preset.

Related errors

AI-assisted analysis of apify/crawlee@dbe57fb09c (2026-08-30). Data as JSON: /api/errors/1604f05543dbc88a. Report an issue: GitHub.