santifer/career-ops · error · Error

Invalid URL

Error message

Invalid URL: ${url}

What it means

assertSafeRemoteUrl is an SSRF defense-in-depth gate run before fetching any job-page URL. When the input cannot be parsed by the URL constructor at all (malformed string, missing scheme, spaces, unencoded characters), it throws 'Invalid URL: <url>'. This is a fail-closed check: the library refuses to guess or repair URLs coming from user-maintained portals.yml / pipeline.md files.

Solutions

  1. Fix the source entry so it is a fully qualified absolute URL including the scheme (https://...)
  2. Trim whitespace, quotes, and markdown artifacts from the value in portals.yml / pipeline.md
  3. Validate the string with new URL() in a scratch script before putting it back into the config
  4. If the value legitimately has no scheme, add 'https://' explicitly rather than relying on defaults

Example fix

// before (portals.yml / pipeline.md entry)
careers_url: acme.com/careers

// after
careers_url: https://acme.com/careers
Defensive patterns

Strategy: validation

Validate before calling

function isValidHttpUrl(s) {
  try {
    const u = new URL(s);
    return u.protocol === 'https:' || u.protocol === 'http:';
  } catch { return false; }
}
// call before fetchJobPage: if (!isValidHttpUrl(entry.url)) skip/fix entry;

Type guard

function isUrlString(v) {
  return typeof v === 'string' && v.length > 0 && new URL(v) instanceof URL ? v : null;
}

Try / catch

let html;
try {
  html = await fetchJobPage(url);
} catch (e) {
  if (e.message.startsWith('Invalid URL:')) {
    console.error(`Fix config entry, not a valid URL: ${e.message}`);
    return null; // skip entry, flag portals.yml/pipeline.md for manual fix
  }
  throw e;
}

Prevention

When it happens

Trigger: fetchJobPage (or any caller of assertSafeRemoteUrl) is given a string that new URL() rejects — e.g. an empty string, a bare hostname without a scheme ('example.com/jobs/123'), a URL with unencoded spaces or invalid characters, or a truncated/garbled entry read from portals.yml or pipeline.md.

Common situations: A hand-edited portals.yml careers_url missing 'https://', a pipeline.md line that wrapped the URL across lines or captured trailing markdown, a copy-paste dropping the scheme, or programmatic concatenation producing 'url:https://...' style prefixes.

Understand the failure class

Background: "Invalid URL" errors: why new URL(), URI.parse, and reqwest::Url reject your string — missing scheme, whitespace, and bad path format — this error's family across 39 libraries.

Related errors

AI-assisted analysis of santifer/career-ops@aac998c7ed (2026-09-16). Data as JSON: /api/errors/b6ead8c3bc3192ad. Report an issue: GitHub.

Appendix: source

Thrown at openrouter-runner.mjs:409

    ctx.profile,
    '---',
    'CV (Markdown):',
    ctx.cv,
    '---',
    'OUTPUT LANGUAGE:',
    languageInstruction,
  ].filter(Boolean).join('\n\n');
}

// ---------------------------------------------------------------------------
// Job page content fetcher (Playwright-first, plain fetch fallback)
// ---------------------------------------------------------------------------
// Reject unsafe fetch targets (SSRF defense-in-depth): http(s) only, never
// loopback / link-local / private / cloud-metadata hosts. URLs come from the
// user's own portals.yml / pipeline.md, but we still fail closed.
function assertSafeRemoteUrl(url) {
  let u;
  try { u = new URL(url); } catch { throw new Error(`Invalid URL: ${url}`); }
  if (u.protocol !== 'https:' && u.protocol !== 'http:') {
    throw new Error(`Refusing non-HTTP(S) URL: ${url}`);
  }
  const host = u.hostname.toLowerCase();
  const blocked = host === 'localhost' || host === '::1' || host.endsWith('.local') ||
    /^127\./.test(host) || /^10\./.test(host) || /^192\.168\./.test(host) ||
    /^169\.254\./.test(host) || /^172\.(1[6-9]|2\d|3[01])\./.test(host);
  if (blocked) throw new Error(`Refusing private/loopback host: ${host}`);
  return u;
}

async function fetchJobPage(url) {
  assertSafeRemoteUrl(url);
  let chromium;
  try {
    ({ chromium } = await import('playwright'));
  } catch {
    console.warn('[fetch] Playwright unavailable — falling back to plain fetch.');

View on GitHub (pinned to aac998c7ed)