{"record":{"id":"831e78bf354b7bb9","repo":"santifer/career-ops","slug":"blocked-request-to-restricted-destination-dns","errorCode":null,"errorMessage":"   Blocked request to restricted destination (DNS): ${requestUrl} - ${err.message}","messagePattern":"   Blocked request to restricted destination \\(DNS\\): (.+?) - (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"archive-posting.mjs","lineNumber":313,"sourceCode":" * catches a public hostname resolving into private space.\n *\n * @param {import('playwright').BrowserContext} context - Context to guard.\n */\nexport async function installEgressGuard(context) {\n  await context.route('**/*', async (route) => {\n    const requestUrl = route.request().url();\n\n    const verdict = rejectPrivateOrInvalid(requestUrl);\n    if (verdict) {\n      console.warn(`   Blocked request to restricted destination: ${requestUrl} (${verdict.reason})`);\n      return route.abort('blockedbyclient');\n    }\n\n    try {\n      await validateUrlSecurity(requestUrl);\n      return route.continue();\n    } catch (err) {\n      console.warn(`   Blocked request to restricted destination (DNS): ${requestUrl} - ${err.message}`);\n      return route.abort('blockedbyclient');\n    }\n  });\n}\n\nexport async function archiveUrl(browser, url, { company: companyHint, role: roleHint } = {}) {\n  console.log(`\\n🔗  ${url}`);\n\n  // Refuse before launching any navigation, so an obviously-internal target\n  // never reaches Playwright at all.\n  const preGuard = rejectPrivateOrInvalid(url);\n  if (preGuard) {\n    throw new Error(`refusing to archive restricted destination: ${preGuard.reason}`);\n  }\n\n  const context = await browser.newContext();\n  await installEgressGuard(context);\n  const page = await context.newPage();","sourceCodeStart":295,"sourceCodeEnd":331,"githubUrl":"https://github.com/santifer/career-ops/blob/aac998c7ed7248ea853b720ceeb1fdbeb322fc5d/archive-posting.mjs#L295-L331","documentation":"This message is emitted by installEgressGuard, a Playwright route interceptor installed on the archival browser context. Before every outgoing request the guard calls validateUrlSecurity(requestUrl); if that check throws (typically because DNS resolution of the destination is disallowed, points at a restricted/private address, or the host is otherwise not permitted), the request is logged with this warning and aborted with 'blockedbyclient'. It is an intentional security guard, not a malfunction — the code is protecting the scraping browser from SSRF-style access to internal or blocked destinations while archiving job postings.","triggerScenarios":"Calling archiveUrl() (or any code that installs the egress guard) and having the browser attempt to load a URL whose host fails validateUrlSecurity: a hostname that does not resolve, resolves to a private/loopback/link-local address, an IP-literal pointing at internal infrastructure, or a host on the configured deny list. The route.continue() path is skipped and route.abort('blockedbyclient') fires instead, printing this warn.","commonSituations":"A job posting URL redirects to or pulls resources from a blocked host (trackers, CDNs on the deny list, internal staging URLs); a JD link contains a dead domain whose DNS lookup fails; a developer tests against localhost or an internal staging server while running the archiver; a typo in a company careers URL (e.g. 'greenhous.xyz') causes resolution failure; running in an environment where DNS for the target host is not available.","solutions":["Inspect the URL in the warning: if it is a redirect target or sub-resource (font/tracker/analytics), it can be safely ignored — only the primary posting URL needs to be reachable.","If the primary posting URL itself is blocked, verify the domain resolves: run `dig <host>` or `nslookup <host>` to confirm DNS works from this machine.","If the host is legitimate but restricted, update the allow/deny configuration consumed by validateUrlSecurity (or its resolver policy) to permit the destination, then retry the archive run.","If the URL points at internal infrastructure (localhost, 10.x, 192.168.x, 169.254.x), use the public externally-resolvable URL instead — the guard intentionally blocks private destinations."],"exampleFix":"// before: archiving an internal/staging posting URL\nawait archiveUrl(browser, 'http://staging.internal.company.com/jobs/123');\n// warn: Blocked request to restricted destination (DNS)\n\n// after: use the public, externally resolvable posting URL\nawait archiveUrl(browser, 'https://company.com/careers/jobs/123');","handlingStrategy":"validation","validationCode":"import { validateUrlSecurity } from './archive-posting.mjs';\n// Check before invoking the archive flow:\ntry {\n  await validateUrlSecurity(url);\n} catch (err) {\n  console.error(`Refusing to archive unreachable/restricted URL ${url}: ${err.message}`);\n  return; // skip this URL instead of letting the route be aborted mid-scrape\n}","typeGuard":"function isPublicHttpUrl(url) {\n  try {\n    const u = new URL(url);\n    return (u.protocol === 'http:' || u.protocol === 'https:') &&\n      !/^(localhost|127\\.|10\\.|192\\.168\\.|169\\.254\\.|172\\.(1[6-9]|2\\d|3[01])\\.)/.test(u.hostname) &&\n      !u.hostname.endsWith('.internal');\n  } catch { return false; }\n}","tryCatchPattern":"try {\n  await archiveUrl(browser, url, { company, role });\n} catch (err) {\n  if (err.message.includes('blockedbyclient') || /restricted destination/i.test(err.message)) {\n    console.warn(`Skipping ${url}: blocked by egress guard — verify the domain is public and resolves.`);\n  } else {\n    throw err;\n  }\n}","preventionTips":["Pre-validate every URL with validateUrlSecurity before feeding it to archiveUrl so blocked destinations fail fast with a clear message.","Only archive externally resolvable public URLs; never point the archiver at localhost or internal staging hosts.","Check company careers URLs for typos before batch runs; a dead domain wastes a browser session.","Treat 'Blocked request' warnings for sub-resources (trackers, fonts) as expected noise and filter them in log review."],"tags":["network","dns","security","playwright","egress"],"backgroundTag":"dns-lookup-failed","analyzedSha":"aac998c7ed7248ea853b720ceeb1fdbeb322fc5d","analyzedAt":"2026-09-16T06:35:29.214Z","contentChangedAt":"2026-09-16T06:35:29.214Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}