santifer/career-ops · error · Error

Refusing non-HTTP(S) URL

Error message

Refusing non-HTTP(S) URL: ${url}

What it means

The second gate in assertSafeRemoteUrl: the string parses as a URL but its protocol is neither https: nor http:. The library only fetches web pages over HTTP(S), so schemes like file:, ftp:, data:, or javascript: are rejected outright as part of SSRF fail-closed hardening.

Solutions

  1. Replace the entry with an https:// URL pointing at the live posting
  2. If you have a local copy of the JD, don't fetch it as a URL — save it as a jds/ capture (e.g. via archive-posting.mjs) and reference local:jds/{file}
  3. Correct typos in the scheme (http// → https://) so the URL parses with a valid protocol
  4. Remove ftp:/data:/javascript: style entries from portals.yml / pipeline.md entirely

Example fix

// before (pipeline.md / portals.yml)
url: file:///tmp/job-posting.html

// after
url: https://jobs.acme.com/postings/12345
Defensive patterns

Strategy: validation

Validate before calling

function isWebUrl(s) {
  try { const u = new URL(s); return u.protocol === 'https:' || u.protocol === 'http:'; }
  catch { return false; }
}
// reject file://, ftp://, data:, etc. before calling fetchJobPage

Type guard

function hasHttpScheme(v) {
  if (typeof v !== 'string') return false;
  const m = v.match(/^https?:\/\//i);
  return m !== null;
}

Try / catch

try {
  const text = await fetchJobPage(url);
} catch (e) {
  if (e.message.startsWith('Refusing non-HTTP(S) URL')) {
    console.error(`Unsupported scheme for ${url} — use an https:// link or a local jds/ capture`);
    return null;
  }
  throw e;
}

Prevention

When it happens

Trigger: fetchJobPage receives a URL whose protocol check fails — e.g. 'file:///home/user/job.html', 'ftp://example.com/posting', 'data:text/html,...', or a config entry like 'localhost.job' with a typo'd scheme such as 'https//example.com' (which parses with protocol 'https:/' malformed... actually parses as scheme 'https' with invalid rest) or 'webcal://...'.

Common situations: Someone pointed a pipeline entry at a locally saved HTML file with file://, an internal tool emitted a custom-scheme deep link, or a copy-paste from a desktop app produced a non-web scheme.

Understand the failure class

Background: "Invalid URL" / "URL cannot be empty": fix the malformed or missing URL behind request-construction failures — this error's family across 50 libraries.

Related errors

AI-assisted analysis of santifer/career-ops@aac998c7ed (2026-09-16). Data as JSON: /api/errors/9f2add13c3b38222. Report an issue: GitHub.

Appendix: source

Thrown at openrouter-runner.mjs:411

    'CV (Markdown):',
    ctx.cv,
    '---',
    'OUTPUT LANGUAGE:',
    languageInstruction,
  ].filter(Boolean).join('\n\n');
}

// ---------------------------------------------------------------------------
// Job page content fetcher (Playwright-first, plain fetch fallback)
// ---------------------------------------------------------------------------
// Reject unsafe fetch targets (SSRF defense-in-depth): http(s) only, never
// loopback / link-local / private / cloud-metadata hosts. URLs come from the
// user's own portals.yml / pipeline.md, but we still fail closed.
function assertSafeRemoteUrl(url) {
  let u;
  try { u = new URL(url); } catch { throw new Error(`Invalid URL: ${url}`); }
  if (u.protocol !== 'https:' && u.protocol !== 'http:') {
    throw new Error(`Refusing non-HTTP(S) URL: ${url}`);
  }
  const host = u.hostname.toLowerCase();
  const blocked = host === 'localhost' || host === '::1' || host.endsWith('.local') ||
    /^127\./.test(host) || /^10\./.test(host) || /^192\.168\./.test(host) ||
    /^169\.254\./.test(host) || /^172\.(1[6-9]|2\d|3[01])\./.test(host);
  if (blocked) throw new Error(`Refusing private/loopback host: ${host}`);
  return u;
}

async function fetchJobPage(url) {
  assertSafeRemoteUrl(url);
  let chromium;
  try {
    ({ chromium } = await import('playwright'));
  } catch {
    console.warn('[fetch] Playwright unavailable — falling back to plain fetch.');
  }

View on GitHub (pinned to aac998c7ed)