apify/crawlee · warning · ContextPipelineInterruptedError

Skipping request (starting url: -> loaded url: ) because it…

Error message

Skipping request ${request.id} (starting url: ${request.url} -> loaded url: ${request.loadedUrl}) because it does not match the enqueue strategy (${request['enqueueStrategy']}).

What it means

After a redirect, the crawler checks whether the final loaded URL still satisfies the request's enqueue strategy (e.g. SAME_DOMAIN, SAME_HOSTNAME). If the redirect crossed outside the allowed scope, the request is marked SKIPPED, recorded as skipped with reason 'redirect', and the pipeline is interrupted with this error.

Solutions

1. Loosen the enqueue strategy (e.g. use EnqueueStrategy.SameDomain instead of SameHostname) if the redirects are legitimate.
2. Pre-resolve redirects with a HEAD/GET request and enqueue the final URL directly.
3. Handle the skipped 'redirect' requests in the skipped-request hook and re-enqueue the loadedUrl with an appropriate strategy.
4. Add the redirect target's domain to the allowed domains in strategy options if it should be followed.

Example fix

// before
const crawler = new CheerioCrawler({ enqueueStrategy: { strategy: 'same-hostname' } });
// after
const crawler = new CheerioCrawler({ enqueueStrategy: { strategy: 'same-domain' } });

// Verify the final URL of a request stays inside the strategy before enqueueing
const finalUrl = await resolveRedirects(request.url); // HEAD-follow helper
if (!isSameDomain(finalUrl, new URL(request.url).hostname)) {
  request.enqueueStrategy = { strategy: 'same-domain' }; // loosen or skip enqueueing
}

function matchesStrategy(loadedUrl, originUrl, strategy) {
  const a = new URL(loadedUrl), b = new URL(originUrl);
  if (strategy === 'same-hostname') return a.hostname === b.hostname;
  if (strategy === 'same-domain') return a.hostname.replace(/^www\./, '') === b.hostname.replace(/^www\./, '');
  return true;
}

try {
  await crawler.run(requests);
} catch (err) {
  if (err.name === 'ContextPipelineInterruptedError' && err.message.includes('enqueue strategy')) {
    // treat request as skipped; optionally re-enqueue request.loadedUrl with a looser strategy
  } else throw err;
}

When it happens

Trigger: A request enqueued with a restrictive enqueueStrategy (e.g. sameHostname) whose server responds with a redirect to a URL that violates that strategy — the loaded URL no longer matches, so the request is skipped.

Common situations: Sites redirecting http->a different subdomain, to a CDN domain, or to a country-specific domain; enqueuing with SAME_HOSTNAME and hitting a www/non-www or short-link redirect.

Related errors

• Skipping request as disallowed by robots.txt
• Too many redirects ( ) while requesting
• Unknown enqueue strategy

AI-assisted analysis of apify/crawlee@dbe57fb09c (2026-08-30). Data as JSON: /api/errors/8a790fe18acdb409. Report an issue: GitHub.

Appendix: source

Thrown at packages/basic-crawler/src/internals/basic-crawler.ts:1594

                ) as unknown as ContextPipeline<CrawlingContext, Context>;
        } else {
            contextPipeline = subclassPipeline;
        }

        contextPipeline = contextPipeline.compose({
            action: async (context) => {
                const { request } = context;
                if (request && !this.requestMatchesEnqueueStrategy(request)) {
                    // eslint-disable-next-line dot-notation
                    const message = `Skipping request ${request.id} (starting url: ${request.url} -> loaded url: ${request.loadedUrl}) because it does not match the enqueue strategy (${request['enqueueStrategy']}).`;
                    this.log.debug(message);

                    request.noRetry = true;
                    request.state = RequestState.SKIPPED;

                    await this.#handleSkippedRequest({ request, reason: 'redirect' });

                    throw new ContextPipelineInterruptedError(message);
                }
                return context;
            },
        });

        return contextPipeline as ContextPipeline<CrawlingContext, ExtendedContext>;
    }

    /**
     * Checks if the given error is a proxy error by comparing its message to a list of known proxy error messages.
     * Used for retrying requests that failed due to proxy errors.
     *
     * @param error The error to check.
     */
    protected isProxyError(error: Error): boolean {
        return ROTATE_PROXY_ERRORS.some((x: string) => (this.getMessageFromError(error) as any)?.includes(x));
    }

View on GitHub (pinned to dbe57fb09c)