apify/crawlee · error · Error

"${domain}" is not a valid hostname. The `domains` option ta

Error message

"${domain}" is not a valid hostname. The `domains` option takes bare hostnames such as "example.com"; an IPv6 address has to be bracketed, as in "[::1]".

What it means

ThrottlingRequestManager validates each entry of the `domains` option by parsing it as `http://<domain>` and reading back the normalized hostname. If URL parsing fails, the string is not a valid bare hostname. Bare hostnames are expected; IPv6 literals must be bracketed.

Source

Thrown at packages/core/src/storages/throttling_request_manager.ts:364

            options.requestManagerOpener ??
            ((idOrAlias, opts) => RequestQueue.open(idOrAlias, opts) as unknown as Promise<T>);
        this.#baseDelayMs = (options.baseDelaySecs ?? 2) * 1000;
        this.#maxDelayMs = (options.maxDelaySecs ?? 60) * 1000;
        this.#maxDomainStallMs = (options.maxDomainStallSecs ?? 900) * 1000;
        this.#minCrawlDelayMs = (options.minCrawlDelaySecs ?? 0) * 1000;
        this.#throttlesEveryDomain = options.domains === 'all';
        this.#throttleBy = options.throttleBy ?? 'hostname';
        this.#maxThrottledDomains = options.maxThrottledDomains ?? 100;
        this.#persistStateKey = options.persistStateKey ?? DEFAULT_PERSIST_STATE_KEY;
        this.log = serviceLocator.getLogger().child({ prefix: 'ThrottlingRequestManager' });

        for (const domain of Array.isArray(options.domains) ? options.domains : []) {
            let hostname: string;
            try {
                // These are bare hostnames, so they only reach `URL` - and with it IDNA - via a synthetic URL.
                hostname = new URL(`http://${domain}`).hostname;
            } catch {
                throw new Error(
                    `"${domain}" is not a valid hostname. The \`domains\` option takes bare hostnames such as ` +
                        `"example.com"; an IPv6 address has to be bracketed, as in "[::1]".`,
                );
            }

            const key = this.#domainKey(hostname);
            this.#listedDomains.add(key);
            this.domainStates.set(key, newDomainState(key));
        }
    }

    /**
     * The key a URL's requests are grouped under - one delay clock and one sub-queue per key.
     *
     * @param hostname A hostname as `URL` reports it, so in punycode and possibly with a root dot.
     */
    #domainKey(hostname: string): string {
        const normalized = normalizeHostname(hostname);

View on GitHub (pinned to dbe57fb09c)

Solutions

Strip scheme, path, port and userinfo so only the bare hostname remains
Bracket IPv6 literals: '[::1]' instead of '::1'
Trim whitespace and remove empty strings from the domains array
Validate entries with `new URL('http://' + d)` before constructing the manager

Example fix

// before
new ThrottlingRequestManager({ domains: ['https://example.com/', '::1'] }); // throws
// after
new ThrottlingRequestManager({ domains: ['example.com', '[::1]'] });

Defensive patterns

Strategy: validation

Validate before calling

function assertBareHostname(d) {
  if (typeof d !== 'string' || !d.trim()) throw new Error('domains entries must be non-empty strings');
  try { new URL(`http://${d}`); } catch { throw new Error(`"${d}" is not a bare hostname (bracket IPv6, e.g. "[::1]")`); }
}
domains.forEach(assertBareHostname);

Type guard

const isBareHostname = (d) => typeof d === 'string' && d.length > 0 && (() => { try { new URL(`http://${d}`); return true; } catch { return false; } })();

Try / catch

try {
  const mgr = new ThrottlingRequestManager({ domains });
} catch (err) {
  if (/is not a valid hostname/.test(String(err))) {
    throw new Error(`Fix domains option: ${err.message}`);
  }
  throw err;
}

Prevention

Store hostnames without scheme/port/path in configuration
Always bracket IPv6 literals
Trim and deduplicate the domains array at config load time
Validate config entries early at startup

When it happens

Trigger: Passing domains entries containing schemes (http://example.com), paths, ports, userinfo, spaces, empty strings, or unbracketed IPv6 (::1) into `new ThrottlingRequestManager({ domains: [...] })`.

Common situations: Copying a full URL into a domains list; pasting an IPv6 address from logs without brackets; typos/whitespace in a hand-edited domain list; accidentally passing an array of URLs instead of hostnames.

Related errors

AI-assisted analysis of apify/crawlee@dbe57fb09c (2026-08-30). Data as JSON: /api/errors/978833fc23f74419. Report an issue: GitHub.