{"record":{"id":"9f2add13c3b38222","repo":"santifer/career-ops","slug":"refusing-non-http-s-url-url","errorCode":null,"errorMessage":"Refusing non-HTTP(S) URL: ${url}","messagePattern":"Refusing non-HTTP\\(S\\) URL: (.+?)","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"openrouter-runner.mjs","lineNumber":388,"sourceCode":"    'CV (Markdown):',\n    ctx.cv,\n    '---',\n    'OUTPUT LANGUAGE:',\n    languageInstruction,\n  ].filter(Boolean).join('\\n\\n');\n}\n\n// ---------------------------------------------------------------------------\n// Job page content fetcher (Playwright-first, plain fetch fallback)\n// ---------------------------------------------------------------------------\n// Reject unsafe fetch targets (SSRF defense-in-depth): http(s) only, never\n// loopback / link-local / private / cloud-metadata hosts. URLs come from the\n// user's own portals.yml / pipeline.md, but we still fail closed.\nfunction assertSafeRemoteUrl(url) {\n  let u;\n  try { u = new URL(url); } catch { throw new Error(`Invalid URL: ${url}`); }\n  if (u.protocol !== 'https:' && u.protocol !== 'http:') {\n    throw new Error(`Refusing non-HTTP(S) URL: ${url}`);\n  }\n  const host = u.hostname.toLowerCase();\n  const blocked = host === 'localhost' || host === '::1' || host.endsWith('.local') ||\n    /^127\\./.test(host) || /^10\\./.test(host) || /^192\\.168\\./.test(host) ||\n    /^169\\.254\\./.test(host) || /^172\\.(1[6-9]|2\\d|3[01])\\./.test(host);\n  if (blocked) throw new Error(`Refusing private/loopback host: ${host}`);\n  return u;\n}\n\nasync function fetchJobPage(url) {\n  assertSafeRemoteUrl(url);\n  let chromium;\n  try {\n    ({ chromium } = await import('playwright'));\n  } catch {\n    console.warn('[fetch] Playwright unavailable — falling back to plain fetch.');\n  }\n","sourceCodeStart":370,"sourceCodeEnd":406,"githubUrl":"https://github.com/santifer/career-ops/blob/9b17a8ac97b398a496b38e423ae24e433b43254f/openrouter-runner.mjs#L370-L406","documentation":"Thrown by assertSafeRemoteUrl() in openrouter-runner.mjs when fetchJobPage() is asked to retrieve a URL whose protocol is neither http: nor https: (e.g. file:, javascript:, data:, ftp:). It is an SSRF defense-in-depth guard: even though URLs come from the user's own portals.yml / pipeline.md, the runner fails closed against any non-HTTP(S) scheme before passing it to Playwright or fetch().","triggerScenarios":"A pipeline.md or portals.yml entry contains a URL with a non-HTTP(S) protocol; calling fetchJobPage('file:///etc/passwd') or fetchJobPage('javascript:alert(1)'); a malformed/copy-pasted URL where the scheme got stripped or replaced.","commonSituations":"A portals.yml api/careers_url value was pasted from an email/tooltip that prefixed it with a non-http scheme; a relative-looking URL was accidentally prefixed with file:; data: URIs used in a test fixture; a typo inserted a stray character before the scheme.","solutions":["Inspect the exact URL in the error message and correct its scheme to http:// or https://.","Find the source entry in data/pipeline.md or portals.yml and fix the offending api/careers_url value.","If you genuinely need a non-HTTP source, do not route it through fetchJobPage — read it locally instead.","Add a unit test feeding the corrected URL through assertSafeRemoteUrl() to confirm it passes."],"exampleFix":"// before\nurl: \"file:///home/me/jd.html\"\n// after\nurl: \"https://careers.example.com/jobs/123\"","handlingStrategy":"validation","validationCode":"function isHttpUrl(s) {\n  let u;\n  try { u = new URL(s); } catch { return false; }\n  return u.protocol === 'http:' || u.protocol === 'https:';\n}\n// before calling fetchJobPage:\nif (!isHttpUrl(url)) throw new Error(`URL must be http(s): ${url}`);","typeGuard":"/** Narrows a string to a valid http/https URL. */\nfunction isHttpUrl(s) {\n  if (typeof s !== 'string') return false;\n  let u;\n  try { u = new URL(s); } catch { return false; }\n  return u.protocol === 'http:' || u.protocol === 'https:';\n}","tryCatchPattern":null,"preventionTips":["Validate URLs at the portals.yml/pipeline.md ingest boundary, not just at fetch time.","Use a linter/schema check that rejects non-http(s) entries in config.","Never pass user/clipboard text straight to fetchJobPage without a URL parse."],"tags":["ssrf","url-validation","network","security","openrouter-runner"],"backgroundTag":null,"analyzedSha":"9b17a8ac97b398a496b38e423ae24e433b43254f","analyzedAt":"2026-08-13T00:48:39.135Z","schemaVersion":2},"datasetVersion":"2026-08-13T04:17:16.726Z"}