{"record":{"id":"8a7f13e9d45393de","repo":"apify/crawlee","slug":"cannot-decide-what-to-purge-before-running-again","errorCode":null,"errorMessage":"Cannot decide what to purge before running again: `sameDomainDelaySecs` paces the request manager you supplied, so the per-domain queues that have to be emptied are the crawler's while the manager underneath them is yours. Say which you want: `run(requests, { purgeRequestQueue: true })` empties both, `false` empties neither.","messagePattern":"Cannot decide what to purge before running again: `sameDomainDelaySecs` paces the request manager you supplied, so the per-domain queues that have to be emptied are the crawler's while the manager underneath them is yours\\. Say which you want: `run\\(requests, (.+?)\\)` empties both, `false` empties neither\\.","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/basic-crawler/src/internals/basic-crawler.ts","lineNumber":1715,"sourceCode":"     */\n    async run(requests?: TypedRequestsLike<Routes>, options?: CrawlerRunOptions): Promise<FinalStatistics> {\n        if (this.running) {\n            throw new Error(\n                'This crawler instance is already running, you can add more requests to it via `crawler.addRequests()`.',\n            );\n        }\n\n        const { purgeRequestQueue, ...addRequestsOptions } = options ?? {};\n\n        if (this.hasFinishedBefore) {\n            // When executing the run method for the second time explicitly,\n            // we need to purge the RQ to allow processing the same requests again — this is important so users can\n            // pass in failed requests back to the `crawler.run()`, otherwise they would be considered as handled and\n            // ignored — as a failed request is still handled.\n            // `purgeRequestQueue` unset purges only storage the crawler opened itself (see `#purgeableExtent`);\n            // `true` also purges a caller-supplied manager, `false` purges nothing.\n            if (purgeRequestQueue === undefined && this.#purgeableExtent === 'ambiguous') {\n                throw new Error(\n                    'Cannot decide what to purge before running again: `sameDomainDelaySecs` paces the request ' +\n                        'manager you supplied, so the per-domain queues that have to be emptied are the ' +\n                        \"crawler's while the manager underneath them is yours. Say which you want: \" +\n                        '`run(requests, { purgeRequestQueue: true })` empties both, `false` empties neither.',\n                );\n            }\n\n            if (purgeRequestQueue !== false && (this.#purgeableExtent === 'all' || purgeRequestQueue === true)) {\n                // One call from the outside in reaches everything the manager wraps, a pacer's per-domain queues\n                // included.\n                await this.requestManager?.purge?.();\n            }\n\n            // A supplied statistics instance keeps whatever state it was handed - only wipe a default we built.\n            await this.#statisticsDep.ifOwned(async (stats) => {\n                stats.reset();\n                await stats.resetStore();\n            });","sourceCodeStart":1697,"sourceCodeEnd":1733,"githubUrl":"https://github.com/apify/crawlee/blob/dbe57fb09ca607ad59dcf998f3925ef9ac3bb26c/packages/basic-crawler/src/internals/basic-crawler.ts#L1697-L1733","documentation":"When the crawler paces domains via `sameDomainDelaySecs`, that delay lives in the request manager. If the user supplied their own requestManager, purging on re-run becomes ambiguous: the crawler-owned per-domain queues vs. the caller's manager. `purgeRequestQueue` left undefined cannot decide, so run() throws and demands an explicit choice.","triggerScenarios":"Calling `crawler.run(requests)` a second time (re-run) on a crawler configured with `sameDomainDelaySecs` and a user-supplied `requestManager`, without passing an explicit `purgeRequestQueue` value.","commonSituations":"Re-running failed requests back through the same crawler instance; retry scripts that call run() repeatedly; switching to a custom requestManager while relying on the default purge behavior.","solutions":["Pass `purgeRequestQueue: true` to run() to purge both the crawler's queues and the supplied manager (full re-run).","Pass `purgeRequestQueue: false` to purge nothing if re-processing queued requests is unwanted.","On the first run of a fresh crawler, avoid the ambiguity by constructing with defaults instead of a supplied requestManager."],"exampleFix":"// before\nawait crawler.run(failedRequests);\n// after\nawait crawler.run(failedRequests, { purgeRequestQueue: true });","handlingStrategy":"validation","validationCode":"const opts = { purgeRequestQueue: true };\nif (crawlerUsesSuppliedManagerWithSameDomainDelay) {\n  if (opts.purgeRequestQueue === undefined) {\n    throw new Error('Specify purgeRequestQueue true/false for this re-run.');\n  }\n}\nawait crawler.run(requests, opts);","typeGuard":"function hasExplicitPurge(o) { return typeof o.purgeRequestQueue === 'boolean'; }","tryCatchPattern":"try {\n  await crawler.run(requests);\n} catch (err) {\n  if (err.message.startsWith('Cannot decide what to purge')) {\n    await crawler.run(requests, { purgeRequestQueue: true });\n  } else throw err;\n}","preventionTips":["Always pass an explicit purgeRequestQueue boolean when re-running a crawler that uses a supplied requestManager with sameDomainDelaySecs.","Use purgeRequestQueue: true when re-enqueuing failed requests so they are not treated as already handled.","Avoid mixing a custom requestManager with sameDomainDelaySecs unless you understand the purge implications.","Wrap repeated crawler.run() invocations in a helper that fixes purge semantics in one place."],"tags":["configuration","purge","request-manager","rerun"],"backgroundTag":"ambiguous-purge-options","analyzedSha":"dbe57fb09ca607ad59dcf998f3925ef9ac3bb26c","analyzedAt":"2026-08-30T22:22:28.328Z","schemaVersion":2},"datasetVersion":"2026-08-30T23:17:21.991Z"}