Invalid or blocked URL after redirect: ${finalRejected.reaso

Error message

Invalid or blocked URL after redirect: ${finalRejected.reason}

What it means

Thrown by the Playwright scraper in batch-evaluate-gemini.mjs after page.goto(). The URL was allowed before navigation and its subresource requests were routed through rejectPrivateOrInvalid, but HTTP redirects happen server-side, so after landing the code re-checks page.url() against the egress guard (liveness-browser.mjs rejectPrivateOrInvalid). If the final URL is a private/loopback/link-local host, a non-http(s) protocol, or unparseable, the error carries the guard's reason (e.g. 'blocked host 10.0.0.5', 'unsupported protocol file:'). This is the SSRF guard working as designed: the queued URL looked public but redirected somewhere forbidden.

Source

Thrown at batch-evaluate-gemini.mjs:190

    throw new Error(`Invalid or blocked URL: ${rejected.reason}`);
  }
  
  const page = await browser.newPage();
  try {
    await page.route('**/*', (route) => {
      const targetUrl = route.request().url();
      const interceptedRejected = rejectPrivateOrInvalid(targetUrl);
      if (interceptedRejected) {
        return route.abort('accessdenied');
      }
      return route.continue();
    });

    await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 30000 });
    
    const finalRejected = rejectPrivateOrInvalid(page.url());
    if (finalRejected) {
      throw new Error(`Invalid or blocked URL after redirect: ${finalRejected.reason}`);
    }

    await page.waitForTimeout(2000); // wait for dynamic content
    const text = await page.evaluate(() => {
      document.querySelectorAll('script, style, noscript, iframe, svg, img').forEach(s => s.remove());
      return document.body.innerText;
    });
    return text.trim();
  } finally {
    await page.close();
  }
}

async function evaluateWithRetry(jdText, retries = 5) {
  if (typeof retries !== 'number' || isNaN(retries) || retries < 1) retries = 1;
  let attempt = 0;
  let delay = 5000;
  while (attempt < retries) {

View on GitHub (pinned to 60398d6549)

Solutions

Read the reason embedded in the message: 'blocked host X' means the redirect landed on a private/reserved address, 'unsupported protocol X' means a non-http(s) scheme, 'invalid URL' means the final location did not parse.
Follow the redirect chain yourself (curl -IL <url> or browser devtools) to see the actual final destination.
If the final destination is legitimately public, replace the pipeline entry with that final URL so no redirect is involved and re-run.
If the destination is genuinely internal/localhost, remove or SKIP the entry; the guard is an SSRF control and must not be bypassed.
Re-run the batch: the per-URL try/catch marks this entry failed and processing continues with the remaining URLs.

Example fix

// before (data/pipeline.md entry that redirects to an internal host)
| 1 | 2026-08-20 | Acme | Senior EE | https://acme.example/track/abc (302 -> http://10.2.3.4/careers) |

// after (queue the final public destination directly)
| 1 | 2026-08-20 | Acme | Senior EE | https://acme.example/careers/123 |

Defensive patterns

Strategy: try-catch

Try / catch

// in the batch loop, per URL:
try {
  const jdText = await scrapeUrl(browser, url);
} catch (err) {
  if (/Invalid or blocked URL after redirect/.test(err.message)) {
    // deterministic SSRF-guard outcome: record reason, never auto-retry
    console.warn(`SKIP ${url}: egress guard -> ${err.message}`);
    continue; // next pipeline entry
  }
  throw err;
}

Prevention

Queue final-destination URLs in data/pipeline.md, not redirect chains or shorteners.
Never queue internal, staging, or localhost URLs - the guard is by design not bypassable.
Treat 'blocked host'/'unsupported protocol' reasons as verdicts about the entry, not bugs to work around.

When it happens

Trigger: A data/pipeline.md entry 301/302-redirects to an internal hostname or private IP (jobs.acme.com -> 10.2.3.4); a shortener resolving to a private address; a redirect chain ending in a non-http(s) scheme; split-horizon DNS where a public name resolves internally on the run host; a queued http://localhost test link.

Common situations: Batch runs over many queued URLs where one entry points at a staging/internal ATS host; corporate VPN/DNS rewriting a public careers domain to an internal IP; pasting test URLs pointing at localhost; a dead tracker redirect that falls back to an internal error page.

Related errors

AI-assisted analysis of santifer/career-ops@60398d6549 (2026-08-20). Data as JSON: /api/errors/656bdbf9490822ec. Report an issue: GitHub.