[fetch] Playwright unavailable — falling back to plain…

Error message

[fetch] Playwright unavailable — falling back to plain fetch.

What it means

fetchJobPage tries to dynamically import Playwright to render pages with a real Chromium browser. When the `playwright` package cannot be resolved (not installed, or browsers not downloaded), it logs this warning and degrades to a plain HTTP fetch of the URL. This is a graceful fallback, not a fatal error: pages that require JavaScript rendering may return incomplete markup when fetched plainly.

Solutions

Install playwright in the project: `npm install playwright` (or `npm i` to restore dependencies).
Download the Chromium browser binary: `npx playwright install chromium` and, on Linux, `npx playwright install-deps chromium`.
If browsers are intentionally unavailable, accept the fallback but verify the target URL is server-rendered (static HTML) so plain fetch returns usable content.
If playwright is installed but still unresolvable, check that the module is a production dependency, not a devDependency, in the environment running the script.

Example fix

// before
// (playwright missing -> warning at runtime)
// after
// package.json
"dependencies": { "playwright": "^1.45.0" }
// then: npx playwright install chromium

Defensive patterns

Strategy: fallback

Validate before calling

let hasPlaywright = false;
try { require.resolve('playwright'); hasPlaywright = true; } catch {}
if (!hasPlaywright) console.warn('[fetch] Playwright missing — plain fetch will not render JS pages');

Type guard

function canRender(html) {
  return typeof html === 'string' && !/__NEXT_DATA__|window.__APOLLO_STATE__/.test(html) && html.length > 500;
}

Try / catch

try {
  ({ chromium } = await import('playwright'));
} catch {
  console.warn('[fetch] Playwright unavailable — falling back to plain fetch.');
}

Prevention

Add playwright as a production dependency and run `npx playwright install chromium` in postinstall/CI setup.
Run `node doctor.mjs` before pipeline runs to catch missing browser installs.
For JS-rendered job boards, treat plain-fetch results with suspicion and validate that expected content markers appear in the HTML.

When it happens

Trigger: Calling fetchJobPage(url) in a Node environment where `import('playwright')` rejects — playwright not in node_modules, only `playwright-core` installed, browsers never downloaded (`npx playwright install` not run), or an install with SKIP_BROWSER_DOWNLOAD.

Common situations: Fresh clone of the repo without `npm install` completing; CI containers without browser system dependencies; production images that prune devDependencies where playwright lives; switching machines without reinstalling browsers.

Understand the failure class

Background: "X is not installed. Please install it with pip install Y": missing optional dependency errors — ImportError/ValueError raised when a library's optional extra was never installed — this error's family across 22 libraries.

Related errors

AI-assisted analysis of santifer/career-ops@aac998c7ed (2026-09-16). Data as JSON: /api/errors/d99f6769cbd70a48. Report an issue: GitHub.

Appendix: source

Thrown at openrouter-runner.mjs:427

  try { u = new URL(url); } catch { throw new Error(`Invalid URL: ${url}`); }
  if (u.protocol !== 'https:' && u.protocol !== 'http:') {
    throw new Error(`Refusing non-HTTP(S) URL: ${url}`);
  }
  const host = u.hostname.toLowerCase();
  const blocked = host === 'localhost' || host === '::1' || host.endsWith('.local') ||
    /^127\./.test(host) || /^10\./.test(host) || /^192\.168\./.test(host) ||
    /^169\.254\./.test(host) || /^172\.(1[6-9]|2\d|3[01])\./.test(host);
  if (blocked) throw new Error(`Refusing private/loopback host: ${host}`);
  return u;
}

async function fetchJobPage(url) {
  assertSafeRemoteUrl(url);
  let chromium;
  try {
    ({ chromium } = await import('playwright'));
  } catch {
    console.warn('[fetch] Playwright unavailable — falling back to plain fetch.');
  }

  if (chromium) {
    let browser;
    try {
      browser = await chromium.launch({ headless: true });
      const page = await browser.newPage();
      await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 30_000 });
      await page.waitForTimeout(2000); // wait for SPA render
      const text = await page.evaluate(() => {
        document.querySelectorAll('script,style,nav,footer,header').forEach(el => el.remove());
        return (document.body?.innerText || document.body?.textContent || '').replace(/\s+/g, ' ').trim();
      });
      return text.slice(0, 16_000);
    } catch (e) {
      console.warn(`[fetch] Playwright error: ${e.message} — falling back to plain fetch.`);
    } finally {
      if (browser) await browser.close().catch(() => {});

View on GitHub (pinned to aac998c7ed)