{"record":{"id":"d99f6769cbd70a48","repo":"santifer/career-ops","slug":"fetch-playwright-unavailable-falling-back-to-p","errorCode":null,"errorMessage":"[fetch] Playwright unavailable — falling back to plain fetch.","messagePattern":"\\[fetch\\] Playwright unavailable — falling back to plain fetch\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"openrouter-runner.mjs","lineNumber":427,"sourceCode":"  try { u = new URL(url); } catch { throw new Error(`Invalid URL: ${url}`); }\n  if (u.protocol !== 'https:' && u.protocol !== 'http:') {\n    throw new Error(`Refusing non-HTTP(S) URL: ${url}`);\n  }\n  const host = u.hostname.toLowerCase();\n  const blocked = host === 'localhost' || host === '::1' || host.endsWith('.local') ||\n    /^127\\./.test(host) || /^10\\./.test(host) || /^192\\.168\\./.test(host) ||\n    /^169\\.254\\./.test(host) || /^172\\.(1[6-9]|2\\d|3[01])\\./.test(host);\n  if (blocked) throw new Error(`Refusing private/loopback host: ${host}`);\n  return u;\n}\n\nasync function fetchJobPage(url) {\n  assertSafeRemoteUrl(url);\n  let chromium;\n  try {\n    ({ chromium } = await import('playwright'));\n  } catch {\n    console.warn('[fetch] Playwright unavailable — falling back to plain fetch.');\n  }\n\n  if (chromium) {\n    let browser;\n    try {\n      browser = await chromium.launch({ headless: true });\n      const page = await browser.newPage();\n      await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 30_000 });\n      await page.waitForTimeout(2000); // wait for SPA render\n      const text = await page.evaluate(() => {\n        document.querySelectorAll('script,style,nav,footer,header').forEach(el => el.remove());\n        return (document.body?.innerText || document.body?.textContent || '').replace(/\\s+/g, ' ').trim();\n      });\n      return text.slice(0, 16_000);\n    } catch (e) {\n      console.warn(`[fetch] Playwright error: ${e.message} — falling back to plain fetch.`);\n    } finally {\n      if (browser) await browser.close().catch(() => {});","sourceCodeStart":409,"sourceCodeEnd":445,"githubUrl":"https://github.com/santifer/career-ops/blob/aac998c7ed7248ea853b720ceeb1fdbeb322fc5d/openrouter-runner.mjs#L409-L445","documentation":"fetchJobPage tries to dynamically import Playwright to render pages with a real Chromium browser. When the `playwright` package cannot be resolved (not installed, or browsers not downloaded), it logs this warning and degrades to a plain HTTP fetch of the URL. This is a graceful fallback, not a fatal error: pages that require JavaScript rendering may return incomplete markup when fetched plainly.","triggerScenarios":"Calling fetchJobPage(url) in a Node environment where `import('playwright')` rejects — playwright not in node_modules, only `playwright-core` installed, browsers never downloaded (`npx playwright install` not run), or an install with SKIP_BROWSER_DOWNLOAD.","commonSituations":"Fresh clone of the repo without `npm install` completing; CI containers without browser system dependencies; production images that prune devDependencies where playwright lives; switching machines without reinstalling browsers.","solutions":["Install playwright in the project: `npm install playwright` (or `npm i` to restore dependencies).","Download the Chromium browser binary: `npx playwright install chromium` and, on Linux, `npx playwright install-deps chromium`.","If browsers are intentionally unavailable, accept the fallback but verify the target URL is server-rendered (static HTML) so plain fetch returns usable content.","If playwright is installed but still unresolvable, check that the module is a production dependency, not a devDependency, in the environment running the script."],"exampleFix":"// before\n// (playwright missing -> warning at runtime)\n// after\n// package.json\n\"dependencies\": { \"playwright\": \"^1.45.0\" }\n// then: npx playwright install chromium","handlingStrategy":"fallback","validationCode":"let hasPlaywright = false;\ntry { require.resolve('playwright'); hasPlaywright = true; } catch {}\nif (!hasPlaywright) console.warn('[fetch] Playwright missing — plain fetch will not render JS pages');","typeGuard":"function canRender(html) {\n  return typeof html === 'string' && !/__NEXT_DATA__|window.__APOLLO_STATE__/.test(html) && html.length > 500;\n}","tryCatchPattern":"try {\n  ({ chromium } = await import('playwright'));\n} catch {\n  console.warn('[fetch] Playwright unavailable — falling back to plain fetch.');\n}","preventionTips":["Add playwright as a production dependency and run `npx playwright install chromium` in postinstall/CI setup.","Run `node doctor.mjs` before pipeline runs to catch missing browser installs.","For JS-rendered job boards, treat plain-fetch results with suspicion and validate that expected content markers appear in the HTML."],"tags":["playwright","browser","fallback","dependency","scraping"],"backgroundTag":"missing-optional-dependency","analyzedSha":"aac998c7ed7248ea853b720ceeb1fdbeb322fc5d","analyzedAt":"2026-09-16T06:35:29.214Z","contentChangedAt":"2026-09-16T06:35:29.214Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}