{"record":{"id":"978833fc23f74419","repo":"apify/crawlee","slug":"domain-is-not-a-valid-hostname-the-domains","errorCode":null,"errorMessage":"\"${domain}\" is not a valid hostname. The `domains` option takes bare hostnames such as \"example.com\"; an IPv6 address has to be bracketed, as in \"[::1]\".","messagePattern":"\"(.+?)\" is not a valid hostname\\. The `domains` option takes bare hostnames such as \"example\\.com\"; an IPv6 address has to be bracketed, as in \"\\[::1\\]\"\\.","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/core/src/storages/throttling_request_manager.ts","lineNumber":364,"sourceCode":"            options.requestManagerOpener ??\n            ((idOrAlias, opts) => RequestQueue.open(idOrAlias, opts) as unknown as Promise<T>);\n        this.#baseDelayMs = (options.baseDelaySecs ?? 2) * 1000;\n        this.#maxDelayMs = (options.maxDelaySecs ?? 60) * 1000;\n        this.#maxDomainStallMs = (options.maxDomainStallSecs ?? 900) * 1000;\n        this.#minCrawlDelayMs = (options.minCrawlDelaySecs ?? 0) * 1000;\n        this.#throttlesEveryDomain = options.domains === 'all';\n        this.#throttleBy = options.throttleBy ?? 'hostname';\n        this.#maxThrottledDomains = options.maxThrottledDomains ?? 100;\n        this.#persistStateKey = options.persistStateKey ?? DEFAULT_PERSIST_STATE_KEY;\n        this.log = serviceLocator.getLogger().child({ prefix: 'ThrottlingRequestManager' });\n\n        for (const domain of Array.isArray(options.domains) ? options.domains : []) {\n            let hostname: string;\n            try {\n                // These are bare hostnames, so they only reach `URL` - and with it IDNA - via a synthetic URL.\n                hostname = new URL(`http://${domain}`).hostname;\n            } catch {\n                throw new Error(\n                    `\"${domain}\" is not a valid hostname. The \\`domains\\` option takes bare hostnames such as ` +\n                        `\"example.com\"; an IPv6 address has to be bracketed, as in \"[::1]\".`,\n                );\n            }\n\n            const key = this.#domainKey(hostname);\n            this.#listedDomains.add(key);\n            this.domainStates.set(key, newDomainState(key));\n        }\n    }\n\n    /**\n     * The key a URL's requests are grouped under - one delay clock and one sub-queue per key.\n     *\n     * @param hostname A hostname as `URL` reports it, so in punycode and possibly with a root dot.\n     */\n    #domainKey(hostname: string): string {\n        const normalized = normalizeHostname(hostname);","sourceCodeStart":346,"sourceCodeEnd":382,"githubUrl":"https://github.com/apify/crawlee/blob/dbe57fb09ca607ad59dcf998f3925ef9ac3bb26c/packages/core/src/storages/throttling_request_manager.ts#L346-L382","documentation":"ThrottlingRequestManager validates each entry of the `domains` option by parsing it as `http://<domain>` and reading back the normalized hostname. If URL parsing fails, the string is not a valid bare hostname. Bare hostnames are expected; IPv6 literals must be bracketed.","triggerScenarios":"Passing domains entries containing schemes (http://example.com), paths, ports, userinfo, spaces, empty strings, or unbracketed IPv6 (::1) into `new ThrottlingRequestManager({ domains: [...] })`.","commonSituations":"Copying a full URL into a domains list; pasting an IPv6 address from logs without brackets; typos/whitespace in a hand-edited domain list; accidentally passing an array of URLs instead of hostnames.","solutions":["Strip scheme, path, port and userinfo so only the bare hostname remains","Bracket IPv6 literals: '[::1]' instead of '::1'","Trim whitespace and remove empty strings from the domains array","Validate entries with `new URL('http://' + d)` before constructing the manager"],"exampleFix":"// before\nnew ThrottlingRequestManager({ domains: ['https://example.com/', '::1'] }); // throws\n// after\nnew ThrottlingRequestManager({ domains: ['example.com', '[::1]'] });","handlingStrategy":"validation","validationCode":"function assertBareHostname(d) {\n  if (typeof d !== 'string' || !d.trim()) throw new Error('domains entries must be non-empty strings');\n  try { new URL(`http://${d}`); } catch { throw new Error(`\"${d}\" is not a bare hostname (bracket IPv6, e.g. \"[::1]\")`); }\n}\ndomains.forEach(assertBareHostname);","typeGuard":"const isBareHostname = (d) => typeof d === 'string' && d.length > 0 && (() => { try { new URL(`http://${d}`); return true; } catch { return false; } })();","tryCatchPattern":"try {\n  const mgr = new ThrottlingRequestManager({ domains });\n} catch (err) {\n  if (/is not a valid hostname/.test(String(err))) {\n    throw new Error(`Fix domains option: ${err.message}`);\n  }\n  throw err;\n}","preventionTips":["Store hostnames without scheme/port/path in configuration","Always bracket IPv6 literals","Trim and deduplicate the domains array at config load time","Validate config entries early at startup"],"tags":["validation","configuration","throttling"],"backgroundTag":"invalid-hostname","analyzedSha":"dbe57fb09ca607ad59dcf998f3925ef9ac3bb26c","analyzedAt":"2026-08-30T22:22:28.328Z","schemaVersion":2},"datasetVersion":"2026-08-30T23:17:21.991Z"}