apify/crawlee · error · Error

The request is not being processed (uniqueKey: ${uniqueKey})

Error message

The request is not being processed (uniqueKey: ${uniqueKey})

What it means

ensureInProgress() checks the list's inProgress set before allowing markRequestAsHandled(). If the uniqueKey is not currently being processed, the request was never fetched, was already handled, or belongs to a different list, so marking it handled is invalid.

Source

Thrown at packages/core/src/storages/request_list.ts:788

        }
    }

    /**
     * Helper function that validates unique key.
     * Throws an error if uniqueKey is not a non-empty string.
     */
    private ensureUniqueKeyValid(uniqueKey: string): void {
        if (typeof uniqueKey !== 'string' || !uniqueKey) {
            throw new Error("Request object's uniqueKey must be a non-empty string");
        }
    }

    /**
     * Checks that a request is currently being processed and throws an error if not.
     */
    private ensureInProgress(uniqueKey: string): void {
        if (!this.inProgress.has(uniqueKey)) {
            throw new Error(`The request is not being processed (uniqueKey: ${uniqueKey})`);
        }
    }

    /**
     * Throws an error if request list wasn't initialized.
     */
    private ensureIsInitialized(): void {
        if (!this.#isInitialized) {
            throw new Error(
                'RequestList is not initialized; you must call "await requestList.initialize()" before using it!',
            );
        }
    }

    /**
     * Returns the total number of unique requests present in the `RequestList`.
     */
    async getTotalCount(): Promise<number> {

View on GitHub (pinned to dbe57fb09c)

Solutions

1. Only call markRequestAsHandled with the exact request object obtained from fetchNextRequest()
2. Guard against double-handling with a local Set of processed uniqueKeys
3. Verify you are using the same RequestList instance that fetched the request
4. Check whether an earlier exception path already marked the request handled

Example fix

// before
await requestList.markRequestAsHandled(someRequest); // may be un-fetched/duplicate
// after
const processed = new Set<string>();
if (!processed.has(someRequest.uniqueKey)) {
    await requestList.markRequestAsHandled(someRequest);
    processed.add(someRequest.uniqueKey);
}

if (!requestList.inProgress?.has(request.uniqueKey)) {
  throw new Error(`Skipping ${request.uniqueKey}: not in progress`);
}
await requestList.markRequestAsHandled(request);

null

try {
  await requestList.markRequestAsHandled(request);
} catch (err) {
  if (err.message.startsWith('The request is not being processed')) {
    logger.warning(`Already handled or never fetched: ${request.uniqueKey}`); // ignore or log
  } else throw err;
}

When it happens

Trigger: Calling markRequestAsHandled() for a request never returned by fetchNextRequest(); calling it twice for the same request; calling it for a request from another RequestList/RequestQueue; state was restored and in-progress entries were pruned.

Common situations: Double-handling in retry logic without tracking; processing requests from two storage sources with one handler; worker restarts losing in-progress bookkeeping; mixing up uniqueKeys of deduplicated requests.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Recoverable state has not yet been loaded - call initialize(
• Cannot clear the state persisted under key '${this.#persistS
• RequestList sources are already loading or were loaded.
• The state object is invalid: nextIndex must be a non-negativ
• The state object is not consistent with RequestList, too few

AI-assisted analysis of apify/crawlee@dbe57fb09c (2026-08-30). Data as JSON: /api/errors/fe24075f54356afa. Report an issue: GitHub.