apify/crawlee · error · Error

An extracted URL: ${href} is relative and options.baseUrl is

Error message

An extracted URL: ${href} is relative and options.baseUrl is not set. Use options.baseUrl in enqueueLinks() to automatically resolve relative URLs.

What it means

`extractUrlsFromPage()` validates each href extracted from the page via a `enqueueLinks` selector. If a href is relative (fails the `^[a-z][a-z0-9+.-]*:` absolute-URL test) and no `baseUrl` option was provided, it throws immediately with a clear message instead of letting the later `Request` constructor fail with a confusing invalid-URL error. `baseUrl` is what tells the crawler how to resolve relative links against the page's origin.

Source

Thrown at packages/browser-crawler/src/internals/browser-crawler.ts:907

    baseUrl: string,
): Promise<string[]> {
    const urls =
        (await page.$$eval(selector, (linkEls: HTMLLinkElement[]) =>
            linkEls.map((link) => link.getAttribute('href')).filter((href) => !!href),
        )) ?? [];
    const [base] = await page.$$eval('base', (els: HTMLLinkElement[]) => els.map((el) => el.getAttribute('href')));
    const absoluteBaseUrl = base && tryAbsoluteURL(base, baseUrl);

    if (absoluteBaseUrl) {
        baseUrl = absoluteBaseUrl;
    }

    return urls
        .map((href: string) => {
            // Throw a meaningful error when only a relative URL would be extracted instead of waiting for the Request to fail later.
            const isHrefAbsolute = /^[a-z][a-z0-9+.-]*:/.test(href); // Grabbed this in 'is-absolute-url' package.
            if (!isHrefAbsolute && !baseUrl) {
                throw new Error(
                    `An extracted URL: ${href} is relative and options.baseUrl is not set. ` +
                        'Use options.baseUrl in enqueueLinks() to automatically resolve relative URLs.',
                );
            }

            return baseUrl ? tryAbsoluteURL(href, baseUrl) : href;
        })
        .filter((href: string | undefined) => !!href);
}

View on GitHub (pinned to dbe57fb09c)

Solutions

  1. Pass `baseUrl` to `enqueueLinks()`: `await enqueueLinks({ baseUrl: request.loadedUrl, ... })` — the loaded request URL is the usual base
  2. Use the convenience `context.enqueueLinks()` inside the requestHandler where `baseUrl` resolution is wired up; on the raw `extractUrlsFromPage` path, always supply the current page URL as third argument
  3. Preprocess hrefs with `new URL(href, pageUrl).href` if you extract links yourself before enqueueing

Example fix

// before
await enqueueLinks({ selector: 'a.product' }); // relative hrefs on page -> throws
// after
await enqueueLinks({
    selector: 'a.product',
    baseUrl: request.loadedUrl ?? request.url,
    strategy: EnqueueStrategy.SameHostname,
});
Defensive patterns

Strategy: validation

Validate before calling

// Normalize hrefs before enqueueing and assert a base is available:
const base = request.loadedUrl ?? request.url;
const hrefs = [...await page.$evalAll('a[href]', els => els.map(e => e.getAttribute('href')))];
for (const href of hrefs) {
    if (!/^[a-z][a-z0-9+.-]*:/i.test(href) && !base) {
        throw new Error(`Relative href "${href}" requires a baseUrl`);
    }
}

Type guard

function isAbsoluteUrl(href: string): boolean {
    return /^[a-z][a-z0-9+.-]*:/i.test(href);
}

Try / catch

try {
    await enqueueLinks({ baseUrl: request.loadedUrl });
} catch (e) {
    if (e instanceof Error && /is relative and options.baseUrl is not set/.test(e.message)) {
        log.warning(`Skipping link extraction without baseUrl: ${e.message}`);
    } else {
        throw e;
    }
}

Prevention

When it happens

Trigger: Calling `enqueueLinks({ selector: 'a' })` (or the crawler's link extraction) without `options.baseUrl` on a page whose anchors use relative hrefs like `/about` or `../page.html`. The page's `<base>` tag is absent or itself relative, so no absolute base can be derived.

Common situations: Scraping sites that use root-relative or relative hrefs in navigation menus (very common); calling `enqueueLinks` manually from `requestHandler` and forgetting `baseUrl`; pages rendered from string templates or file:// contexts where relative paths abound.

Related errors

AI-assisted analysis of apify/crawlee@dbe57fb09c (2026-08-30). Data as JSON: /api/errors/004dc0d76493449c. Report an issue: GitHub.