apify/crawlee · error · Error

Request options are not valid, the 'id' property must not be

Error message

Request options are not valid, the 'id' property must not be present. Input: ${inspect(opts)}

What it means

In RequestQueue, a request's identity is derived from its url (or uniqueKey); callers must not supply an explicit id. generateRequests() rejects any options object containing a defined id property, since injected ids would collide with the queue's internal id generation and break deduplication/lookup.

Source

Thrown at packages/core/src/storages/request_queue.ts:604

        parseArgument(requests, iterableSchema);

        const { forefront, waitForAllRequestsToBeAdded, batchSize, waitBetweenBatchesMillis, maxNewRequests } =
            parseArgument(options, addRequestsBatchedOptionsSchema);

        const addRequest = this.addRequest.bind(this);

        async function* generateRequests() {
            for await (const opts of requests) {
                // Validate the input
                if (typeof opts === 'object' && opts !== null) {
                    if (opts.url !== undefined && typeof opts.url !== 'string') {
                        throw new Error(
                            `Request options are not valid, the 'url' property is not a string. Input: ${inspect(opts)}`,
                        );
                    }

                    if (opts.id !== undefined) {
                        throw new Error(
                            `Request options are not valid, the 'id' property must not be present. Input: ${inspect(opts)}`,
                        );
                    }

                    if (
                        (opts as any).requestsFromUrl !== undefined &&
                        typeof (opts as any).requestsFromUrl !== 'string'
                    ) {
                        throw new Error(
                            `Request options are not valid, the 'requestsFromUrl' property is not a string. Input: ${inspect(opts)}`,
                        );
                    }
                }

                if (opts && typeof opts === 'object' && 'requestsFromUrl' in opts) {
                    // Handle URL lists right away
                    await addRequest(opts, { forefront });
                } else {

View on GitHub (pinned to dbe57fb09c)

Solutions

Remove the id property before adding: pass { url, ...rest } without id
If re-enqueuing fetched requests, reconstruct from url/uniqueKey or strip id via destructuring
Use uniqueKey (not id) to control deduplication in RequestQueue

Example fix

// before
await queue.addRequestsBatched(fetchedRequests); // items contain id
// after
const clean = fetchedRequests.map(({ id, ...rest }) => rest);
await queue.addRequestsBatched(clean);

Defensive patterns

Strategy: validation

Validate before calling

const withoutIds = requests.map(({ id, ...rest }) => rest);
await queue.addRequestsBatched(withoutIds);

Type guard

function hasNoId(r) {
  return !(r && typeof r === 'object' && 'id' in r && r.id !== undefined);
}

Try / catch

try {
  await queue.addRequestsBatched(requests);
} catch (err) {
  if (err.message.includes("'id' property must not be present")) {
    await queue.addRequestsBatched(requests.map(({ id, ...rest }) => rest));
  } else throw err;
}

Prevention

Never pass queue-fetched Request objects back into addRequestsBatched unchanged
Use uniqueKey for deduplication control, not id
Strip storage-managed fields when re-enqueuing

When it happens

Trigger: Passing { id: '...', url } objects to addRequestsBatched; forwarding Request-like objects (which carry an id) from another queue's output directly into addRequestsBatched; copying stored queue items back into the queue unchanged.

Common situations: Re-enqueuing requests fetched from the same or another RequestQueue (their id field is set); persisting and reloading raw queue entries; adapting code from RequestList (which uses uniqueKey, not id) to RequestQueue.

Related errors

AI-assisted analysis of apify/crawlee@dbe57fb09c (2026-08-30). Data as JSON: /api/errors/15331b030bc6b8c2. Report an issue: GitHub.