santifer/career-ops · warning

[fetch] Playwright error: ${e.message} — falling back to pla

Error message

[fetch] Playwright error: ${e.message} — falling back to plain fetch.

What it means

The dynamic-render fetch path in openrouter-runner.mjs failed inside Playwright and the code falls back to a plain HTTP fetch with a browser-like User-Agent. The failing call is either chromium.launch() (browser binaries never installed, or sandbox error when running as root in a container) or page.goto()/page.evaluate (30s domcontentloaded timeout, DNS failure, or a bot wall). The fallback works but returns raw HTML only — no JS execution — so SPA job pages can come back as an empty shell.

Source

Thrown at openrouter-runner.mjs:420

    ({ chromium } = await import('playwright'));
  } catch {
    console.warn('[fetch] Playwright unavailable — falling back to plain fetch.');
  }

  if (chromium) {
    let browser;
    try {
      browser = await chromium.launch({ headless: true });
      const page = await browser.newPage();
      await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 30_000 });
      await page.waitForTimeout(2000); // wait for SPA render
      const text = await page.evaluate(() => {
        document.querySelectorAll('script,style,nav,footer,header').forEach(el => el.remove());
        return (document.body?.innerText || document.body?.textContent || '').replace(/\s+/g, ' ').trim();
      });
      return text.slice(0, 16_000);
    } catch (e) {
      console.warn(`[fetch] Playwright error: ${e.message} — falling back to plain fetch.`);
    } finally {
      if (browser) await browser.close().catch(() => {});
    }
  }

  // Plain HTTP fallback
  try {
    const r = await fetch(url, {
      headers: { 'User-Agent': DEFAULT_USER_AGENT }
    });
    if (!r.ok) throw new Error(`HTTP ${r.status} ${r.statusText}`);
    const html = await r.text();
    return html.replace(/<[^>]+>/g, ' ').replace(/\s+/g, ' ').trim().slice(0, 16_000);
  } catch (e) {
    throw new Error(`Could not fetch job page: ${e.message}`);
  }
}

View on GitHub (pinned to 60398d6549)

Solutions

1. Run npx playwright install chromium so the primary path works and the fallback is not load-bearing.
2. In root containers, launch with args: ['--no-sandbox'] or run as a non-root user.
3. If pages time out at 30s, raise the goto timeout for the slow hosts you care about.
4. If the fallback's text looks empty, the page is an SPA — fetch the underlying API/JSON the page uses, or capture the JD another way.

Example fix

// before
browser = await chromium.launch({ headless: true });
await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 30_000 });
// after — tolerate root containers and slow origins
browser = await chromium.launch({
  headless: true,
  args: process.getuid?.() === 0 ? ['--no-sandbox'] : [],
});
await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 60_000 });

import { existsSync } from 'node:fs';
import { chromium } from 'playwright';
if (!existsSync(chromium.executablePath())) {
  throw new Error('Chromium not installed — run: npx playwright install chromium');
}

try {
  return await renderWithPlaywright(url);
} catch (e) {
  console.warn(`Playwright failed (${e.message}) — plain-fetch fallback`);
  return await plainFetch(url); // keep the fallback strictly lesser: no JS, same UA, explicit timeout
}

When it happens

Trigger: npx playwright install never run ('Executable doesn't exist at ...'); Docker as root without --no-sandbox; page.goto exceeding the 30s timeout on slow portals; ERR_NAME_NOT_RESOLVED; Cloudflare-type interstitials defeating headless Chromium.

Common situations: Fresh CI containers without cached browser binaries; headless scraping of protected job boards; slow or geographically distant origins.

Related errors

• Playwright extraction failed or blocked, trying fallback Web
• Extracted text too short (likely blocked or empty)
• Image failed to decode within 10s (unreadable or corrupt fil
• Could not fetch job page: ${e.message}
• [fetch] Playwright unavailable — falling back to plain fetch

AI-assisted analysis of santifer/career-ops@60398d6549 (2026-08-20). Data as JSON: /api/errors/a3ce124229a33401. Report an issue: GitHub.