apify/crawlee · error

An extracted URL: ${href} is relative and baseUrl is not set

Error message

An extracted URL: ${href} is relative and baseUrl is not set. Provide a baseUrl to automatically resolve relative URLs.

What it means

extractUrlsFromCheerio validates every extracted href against the is-absolute-url scheme regex. If a href is relative and no baseUrl option was provided, it throws immediately instead of letting the resulting Request fail later with a confusing error. Pass baseUrl so relative URLs are resolved against it via tryAbsoluteURL.

Source

Thrown at packages/utils/src/internals/cheerio.ts:110

 * @return An array of absolute URLs
 */
export function extractUrlsFromCheerio($: CheerioAPI, selector = 'a', baseUrl = ''): string[] {
    const base = $('base').attr('href');
    const absoluteBaseUrl = base && tryAbsoluteURL(base, baseUrl);

    if (absoluteBaseUrl) {
        baseUrl = absoluteBaseUrl;
    }

    return $(selector)
        .map((_i, el) => $(el).attr('href'))
        .get()
        .filter(Boolean)
        .map((href) => {
            // Throw a meaningful error when only a relative URL would be extracted instead of waiting for the Request to fail later.
            const isHrefAbsolute = /^[a-z][a-z0-9+.-]*:/.test(href); // Grabbed this in 'is-absolute-url' package.
            if (!isHrefAbsolute && !baseUrl) {
                throw new Error(
                    `An extracted URL: ${href} is relative and baseUrl is not set. ` +
                        'Provide a baseUrl to automatically resolve relative URLs.',
                );
            }
            return baseUrl ? tryAbsoluteURL(href, baseUrl) : href;
        })
        .filter(Boolean) as string[];
}

View on GitHub (pinned to dbe57fb09c)

Solutions

1. Set the baseUrl option in extractLinks()/urls() so relative URLs resolve
2. Ensure the crawler's Request has a loadedUrl / referer you can pass as baseUrl
3. Filter out relative hrefs before extraction if you only want absolute URLs
4. Use the enqueue strategy with a known origin URL instead of raw extraction

Example fix

// before
await extractLinks({ request, page, selector: 'a' });

// after
await extractLinks({ request, page, selector: 'a', baseUrl: request.loadedUrl });

function ensureAbsoluteOrProvideBase(href: string, baseUrl?: string): boolean {
  const isAbsolute = /^[a-z][a-z0-9+.-]*:/.test(href);
  return isAbsolute || Boolean(baseUrl);
}

function hasBaseUrl(o: { baseUrl?: string }): o is { baseUrl: string } {
  return typeof o.baseUrl === 'string' && o.baseUrl.length > 0;
}

try {
  links = await extractLinks({ request, page, selector: 'a', baseUrl: request.loadedUrl });
} catch (err) {
  if (String(err).includes('baseUrl is not set')) links = [];
  else throw err;
}

When it happens

Trigger: Calling extractLinks()/urls() on a Cheerio page whose anchors include relative hrefs (e.g. '/about', 'page.html') while omitting the baseUrl option.

Common situations: Crawling a site whose HTML uses relative links and forgetting baseUrl in the crawler request options; scraping saved/transformed HTML where the base differs from the source URL; copying an extractLinks call between projects without porting the baseUrl config.

Related errors

• An extracted URL: ${href} is relative and options.baseUrl is
• The `parseWithCheerio` method is not available - `skipNaviga
• Selector '${selector}' not found.
• Selector '${selector}' not found.

AI-assisted analysis of apify/crawlee@dbe57fb09c (2026-08-30). Data as JSON: /api/errors/c5c1bcc503b0092f. Report an issue: GitHub.