santifer/career-ops · warning

[fetch] Playwright error

Error message

[fetch] Playwright error: ${e.message} — falling back to plain fetch.

What it means

Fallback warning in openrouter-runner.mjs's fetchJobPage: the Playwright headless-browser extraction of a job page threw (launch failure, navigation timeout, page.evaluate error, or browser crashed), and the runner logs this message and falls through to a plain HTTP fetch of the same URL.

Solutions

  1. Install browser binaries: npx playwright install chromium
  2. Read err.message: timeout → increase goto timeout or use waitUntil:'domcontentloaded'; missing body → guard `document.body?.innerText`
  3. If Playwright is unavailable, ensure the plain-fetch fallback yields usable text and expect degraded extraction for JS-heavy pages
  4. For blocked pages, set a realistic user agent or run headed mode

Example fix

// before
const text = await page.evaluate(() => {
  document.querySelectorAll('script,style,nav,footer,header').forEach(el => el.remove());
  return (document.body?.innerText || document.body?.textContent || '').replace(/\s+/g, ' ').trim();
});
// after
const text = await page.evaluate(() => {
  document.querySelectorAll('script,style,nav,footer,header').forEach(el => el.remove());
  return ((document.body?.innerText ?? document.body?.textContent) || '').replace(/\s+/g, ' ').trim();
}); // plus try/catch around page.evaluate to return '' instead of throwing
Defensive patterns

Strategy: fallback

Validate before calling

import { chromium } from 'playwright';
try { const b = await chromium.launch(); await b.close(); } catch { console.warn('Playwright browsers not installed; run: npx playwright install chromium'); }

Type guard

function browserAvailable(mod) { try { mod.chromium.executablePath(); return true; } catch { return false; } }

Try / catch

let text; try { text = await extractWithPlaywright(url); } catch (e) { console.warn(`[fetch] Playwright error: ${e.message} — falling back`); text = await plainFetch(url); }

Prevention

When it happens

Trigger: page.goto times out (30s-ish, heavy JS pages), Playwright browser binary not installed, evaluate throws because document.body is null after removing script/nav elements, or browser.close fails after a crash.

Common situations: CI/container without `npx playwright install` browsers; JS-rendered SPA job boards that block headless Chrome; sites with anti-bot measures; slow network causing goto timeout.

Related errors

AI-assisted analysis of santifer/career-ops@aac998c7ed (2026-09-16). Data as JSON: /api/errors/a3ce124229a33401. Report an issue: GitHub.

Appendix: source

Thrown at openrouter-runner.mjs:443

    ({ chromium } = await import('playwright'));
  } catch {
    console.warn('[fetch] Playwright unavailable — falling back to plain fetch.');
  }

  if (chromium) {
    let browser;
    try {
      browser = await chromium.launch({ headless: true });
      const page = await browser.newPage();
      await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 30_000 });
      await page.waitForTimeout(2000); // wait for SPA render
      const text = await page.evaluate(() => {
        document.querySelectorAll('script,style,nav,footer,header').forEach(el => el.remove());
        return (document.body?.innerText || document.body?.textContent || '').replace(/\s+/g, ' ').trim();
      });
      return text.slice(0, 16_000);
    } catch (e) {
      console.warn(`[fetch] Playwright error: ${e.message} — falling back to plain fetch.`);
    } finally {
      if (browser) await browser.close().catch(() => {});
    }
  }

  // Plain HTTP fallback
  try {
    const r = await fetch(url, {
      headers: { 'User-Agent': DEFAULT_USER_AGENT }
    });
    if (!r.ok) throw new Error(`HTTP ${r.status} ${r.statusText}`);
    const html = await r.text();
    return html.replace(/<[^>]+>/g, ' ').replace(/\s+/g, ' ').trim().slice(0, 16_000);
  } catch (e) {
    throw new Error(`Could not fetch job page: ${e.message}`);
  }
}

View on GitHub (pinned to aac998c7ed)