{"record":{"id":"d99f6769cbd70a48","repo":"santifer/career-ops","slug":"fetch-playwright-unavailable-falling-back-to-p","errorCode":null,"errorMessage":"[fetch] Playwright unavailable — falling back to plain fetch.","messagePattern":"\\[fetch\\] Playwright unavailable — falling back to plain fetch\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"openrouter-runner.mjs","lineNumber":404,"sourceCode":"  try { u = new URL(url); } catch { throw new Error(`Invalid URL: ${url}`); }\n  if (u.protocol !== 'https:' && u.protocol !== 'http:') {\n    throw new Error(`Refusing non-HTTP(S) URL: ${url}`);\n  }\n  const host = u.hostname.toLowerCase();\n  const blocked = host === 'localhost' || host === '::1' || host.endsWith('.local') ||\n    /^127\\./.test(host) || /^10\\./.test(host) || /^192\\.168\\./.test(host) ||\n    /^169\\.254\\./.test(host) || /^172\\.(1[6-9]|2\\d|3[01])\\./.test(host);\n  if (blocked) throw new Error(`Refusing private/loopback host: ${host}`);\n  return u;\n}\n\nasync function fetchJobPage(url) {\n  assertSafeRemoteUrl(url);\n  let chromium;\n  try {\n    ({ chromium } = await import('playwright'));\n  } catch {\n    console.warn('[fetch] Playwright unavailable — falling back to plain fetch.');\n  }\n\n  if (chromium) {\n    let browser;\n    try {\n      browser = await chromium.launch({ headless: true });\n      const page = await browser.newPage();\n      await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 30_000 });\n      await page.waitForTimeout(2000); // wait for SPA render\n      const text = await page.evaluate(() => {\n        document.querySelectorAll('script,style,nav,footer,header').forEach(el => el.remove());\n        return (document.body?.innerText || document.body?.textContent || '').replace(/\\s+/g, ' ').trim();\n      });\n      return text.slice(0, 16_000);\n    } catch (e) {\n      console.warn(`[fetch] Playwright error: ${e.message} — falling back to plain fetch.`);\n    } finally {\n      if (browser) await browser.close().catch(() => {});","sourceCodeStart":386,"sourceCodeEnd":422,"githubUrl":"https://github.com/santifer/career-ops/blob/60398d6549a46f5266929538af21cfab94badc75/openrouter-runner.mjs#L386-L422","documentation":"fetchJobPage() in openrouter-runner.mjs prefers Playwright chromium to render job pages (SPAs need JS execution), after assertSafeRemoteUrl() blocks private/loopback hosts. The dynamic import('playwright') is wrapped in try/catch: if the package is not installed in this checkout, it warns and falls back to plain fetch, which often returns an empty JS shell for client-side-rendered ATS pages -- degrading the downstream evaluation input.","triggerScenarios":"npm install run without playwright in dependencies; a stripped-down/global node_modules that lacks playwright; module resolution blocked by NODE_PATH or policy; playwright present but the import itself throws before browser launch.","commonSituations":"CI images omitting browser packages to save size; minimal local installs; container deploys where the fallback is acceptable only for server-rendered pages.","solutions":["Install the dependency: npm install playwright && npx playwright install chromium in the repo root.","Verify the import resolves: node -e \"import('playwright').then(()=>console.log('ok'),e=>console.error(e.message))\".","If a fetch fallback already produced a contentless capture, re-run the evaluation for that URL after installing."],"exampleFix":"# before\n[fetch] Playwright unavailable — falling back to plain fetch.\n# after\n$ npm install playwright\n$ npx playwright install chromium\n$ node openrouter-runner.mjs eval https://example.com/jobs/1","handlingStrategy":"fallback","validationCode":"// Detect the degraded mode up front and refuse URLs that need JS rendering\nlet playwrightOk = false;\ntry { await import('playwright'); playwrightOk = true; } catch {}\nif (!playwrightOk && /lever\\.co|greenhouse\\.io\\/job\\/app/.test(url)) {\n  throw new Error('SPA posting needs Playwright — npm i playwright && npx playwright install chromium');\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Install playwright plus the chromium browser binary in every environment that scrapes job pages.","Verify with node -e \"import('playwright').then(()=>0,e=>{console.error(e.message);process.exit(1)})\" before batch runs.","Remember the fallback also skips the anti-loopback guard's richer checks — plain fetch serves raw HTML only; inspect captures from fallback runs before evaluating them."],"tags":["playwright","dynamic-import","scraping","fetch-fallback","spa-rendering"],"backgroundTag":"optional-dependency-missing","analyzedSha":"60398d6549a46f5266929538af21cfab94badc75","analyzedAt":"2026-08-20T23:00:06.764Z","schemaVersion":2},"datasetVersion":"2026-08-21T18:17:14.833Z"}