{"record":{"id":"501fc82634073134","repo":"apify/crawlee","slug":"the-request-is-not-being-processed-url-url","errorCode":null,"errorMessage":"The request is not being processed (url: ${url})","messagePattern":"The request is not being processed \\(url: (.+?)\\)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/core/src/storages/sitemap_request_loader.ts","lineNumber":635,"sourceCode":"        this.#events.off(EventType.PERSIST_STATE, this.persistState);\n        await this.persistState();\n\n        this.#urlQueueStream.emit('readdata'); // unblocks the potentially waiting `pushNextUrl` call\n    }\n\n    /**\n     * @inheritDoc\n     */\n    async markRequestAsHandled(request: Request): Promise<void> {\n        this.#handledUrlCount += 1;\n        this.ensureInProgress(request.url);\n        this.inProgress.delete(request.url);\n        this.#requestData.delete(request.url);\n    }\n\n    private ensureInProgress(url: string): void {\n        if (!this.inProgress.has(url)) {\n            throw new Error(`The request is not being processed (url: ${url})`);\n        }\n    }\n}\n","sourceCodeStart":617,"sourceCodeEnd":639,"githubUrl":"https://github.com/apify/crawlee/blob/dbe57fb09ca607ad59dcf998f3925ef9ac3bb26c/packages/core/src/storages/sitemap_request_loader.ts#L617-L639","documentation":"SitemapRequestLoader.enforceInProgress is an internal invariant check: it is thrown when markRequestAsHandled (or a similar completion API) is invoked for a URL that the loader does not currently have marked as in-progress. It signals a lifecycle violation — the request was never started, was already handled, or the loader state was reset.","triggerScenarios":"Calling markRequestAsHandled with a request whose URL was never started by the loader, marking the same URL handled twice, or handling a request after the sitemap loader was reinitialized/cleared.","commonSituations":"Custom crawlers manually calling markRequestAsHandled outside the loader's normal processing loop; re-enqueueing a handled request object; concurrency bugs where two workers process the same sitemap URL.","solutions":["Only call markRequestAsHandled for requests returned by the loader's own fetch/next flow","Guard against double-handling by tracking processed URLs in your code","Re-add the request to the queue instead of marking an unknown URL as handled","Check for loader reset (reinit) between starting and handling requests"],"exampleFix":"// before\nloader.markRequestAsHandled(request); // may throw if not started\n// after\ntry {\n  loader.markRequestAsHandled(request);\n} catch (err) {\n  if (!String(err).includes('not being processed')) throw err;\n  loader.inProgressRequests?.add(request.url); // or re-fetch/restart handling\n}","handlingStrategy":"try-catch","validationCode":"null","typeGuard":"null","tryCatchPattern":"try {\n  loader.markRequestAsHandled(request);\n} catch (err) {\n  if (!/not being processed/.test(String(err))) throw err;\n  logger.warning('Attempted to handle untracked sitemap request', { url: request.url });\n}","preventionTips":["Only pass requests obtained from the loader's own iteration to markRequestAsHandled","Track handled URLs in a Set to avoid double-marking","Re-check loader state after resets/reinitialization","Avoid calling internal loader APIs from concurrent tasks"],"tags":["state","internal-invariant","sitemap"],"backgroundTag":"request-not-in-progress","analyzedSha":"dbe57fb09ca607ad59dcf998f3925ef9ac3bb26c","analyzedAt":"2026-08-30T22:22:28.328Z","schemaVersion":2},"datasetVersion":"2026-08-30T23:17:21.991Z"}