santifer/career-ops · error

Invalid or blocked URL: ${rejected.reason}

Error message

Invalid or blocked URL: ${rejected.reason}

What it means

Thrown by scrapeUrl() in batch-evaluate-gemini.mjs, the batch worker's Playwright scraper, when rejectPrivateOrInvalid() (the shared SSRF/egress guard from liveness-browser.mjs) rejects the URL before navigation: unparseable URL, non-http(s) protocol, or a private/loopback host (localhost, 127/8, 10/8, 172.16-31, 192.168/16, 169.254/16, ::1, ::, fc00::/7, fe80::, ::ffff: mapped forms). The same guard is also installed per-request so subresource and redirect requests to private ranges are aborted with 'accessdenied'.

Source

Thrown at batch-evaluate-gemini.mjs:172

═══════════════════════════════════════════════════════
1. You do NOT have access to WebSearch, Playwright, or file writing tools.
2. Generate Blocks A through G in full, in English.
3. Output a machine-readable summary block in this exact format:

---SCORE_SUMMARY---
COMPANY: <company name>
ROLE: <role title>
SCORE: <global score as decimal, e.g. 3.8>
ARCHETYPE: <detected archetype>
LEGITIMACY: <High Confidence | Proceed with Caution | Suspicious>
---END_SUMMARY---
`;
}

async function scrapeUrl(browser, url) {
  const rejected = rejectPrivateOrInvalid(url);
  if (rejected) {
    throw new Error(`Invalid or blocked URL: ${rejected.reason}`);
  }
  
  const page = await browser.newPage();
  try {
    await page.route('**/*', (route) => {
      const targetUrl = route.request().url();
      const interceptedRejected = rejectPrivateOrInvalid(targetUrl);
      if (interceptedRejected) {
        return route.abort('accessdenied');
      }
      return route.continue();
    });

    await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 30000 });
    
    const finalRejected = rejectPrivateOrInvalid(page.url());
    if (finalRejected) {
      throw new Error(`Invalid or blocked URL after redirect: ${finalRejected.reason}`);

View on GitHub (pinned to 60398d6549)

Solutions

  1. Remove or fix the offending entries in the batch input: public https URLs only
  2. Re-add the scheme to bare-domain entries: https://company.com/careers/123
  3. For internal-only postings, archive them manually into jds/ and evaluate from the capture instead of scraping
  4. Pre-filter the list before the run: node -e with rejectPrivateOrInvalid() from liveness-browser.mjs to report all offenders at once

Example fix

# before (batch input entry)
http://10.1.2.3/ats/job/9317

# after (batch input entry)
https://ats.acme-public.com/job/9317
Defensive patterns

Strategy: validation

Validate before calling

import { rejectPrivateOrInvalid } from './liveness-browser.mjs';
function partitionUrls(urls) {
  const ok = [], blocked = [];
  for (const u of urls) {
    (rejectPrivateOrInvalid(u) === null ? ok : blocked).push(u);
  }
  return { ok, blocked };
}
// run the batch with ok; surface blocked to the user instead of failing mid-batch

Type guard

function isScrapableUrl(url) {
  try { return rejectPrivateOrInvalid(url) === null; } catch { return false; }
}

Try / catch

try {
  await scrapeUrl(browser, url);
} catch (e) {
  if (/Invalid or blocked URL/.test(e.message)) {
    results.push({ url, error: 'blocked-or-invalid-url', skipped: true });
    continue; // keep the batch going
  }
  throw e;
}

Prevention

When it happens

Trigger: A batch input/queue containing `http://localhost:8080/jobs`, an intranet ATS URL, `ftp://…`, or a scheme-less string like 'company.com/careers/123'. Each URL in the batch file is scraped in turn, so one bad entry aborts that entry's evaluation.

Common situations: Bulk queue assembled from mixed sources including dev/staging links; a teammate's exported list with internal URLs; entries pasted without the https:// scheme; retrying a batch originally captured on a VPN-only network.

Related errors

AI-assisted analysis of santifer/career-ops@60398d6549 (2026-08-20). Data as JSON: /api/errors/5d4d7f7c9d6d122e. Report an issue: GitHub.