{"record":{"id":"a3ce124229a33401","repo":"santifer/career-ops","slug":"fetch-playwright-error-e-message-falling-b","errorCode":null,"errorMessage":"[fetch] Playwright error: ${e.message} — falling back to plain fetch.","messagePattern":"\\[fetch\\] Playwright error: (.+?) — falling back to plain fetch\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"openrouter-runner.mjs","lineNumber":420,"sourceCode":"    ({ chromium } = await import('playwright'));\n  } catch {\n    console.warn('[fetch] Playwright unavailable — falling back to plain fetch.');\n  }\n\n  if (chromium) {\n    let browser;\n    try {\n      browser = await chromium.launch({ headless: true });\n      const page = await browser.newPage();\n      await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 30_000 });\n      await page.waitForTimeout(2000); // wait for SPA render\n      const text = await page.evaluate(() => {\n        document.querySelectorAll('script,style,nav,footer,header').forEach(el => el.remove());\n        return (document.body?.innerText || document.body?.textContent || '').replace(/\\s+/g, ' ').trim();\n      });\n      return text.slice(0, 16_000);\n    } catch (e) {\n      console.warn(`[fetch] Playwright error: ${e.message} — falling back to plain fetch.`);\n    } finally {\n      if (browser) await browser.close().catch(() => {});\n    }\n  }\n\n  // Plain HTTP fallback\n  try {\n    const r = await fetch(url, {\n      headers: { 'User-Agent': DEFAULT_USER_AGENT }\n    });\n    if (!r.ok) throw new Error(`HTTP ${r.status} ${r.statusText}`);\n    const html = await r.text();\n    return html.replace(/<[^>]+>/g, ' ').replace(/\\s+/g, ' ').trim().slice(0, 16_000);\n  } catch (e) {\n    throw new Error(`Could not fetch job page: ${e.message}`);\n  }\n}\n","sourceCodeStart":402,"sourceCodeEnd":438,"githubUrl":"https://github.com/santifer/career-ops/blob/60398d6549a46f5266929538af21cfab94badc75/openrouter-runner.mjs#L402-L438","documentation":"The dynamic-render fetch path in openrouter-runner.mjs failed inside Playwright and the code falls back to a plain HTTP fetch with a browser-like User-Agent. The failing call is either chromium.launch() (browser binaries never installed, or sandbox error when running as root in a container) or page.goto()/page.evaluate (30s domcontentloaded timeout, DNS failure, or a bot wall). The fallback works but returns raw HTML only — no JS execution — so SPA job pages can come back as an empty shell.","triggerScenarios":"npx playwright install never run ('Executable doesn't exist at ...'); Docker as root without --no-sandbox; page.goto exceeding the 30s timeout on slow portals; ERR_NAME_NOT_RESOLVED; Cloudflare-type interstitials defeating headless Chromium.","commonSituations":"Fresh CI containers without cached browser binaries; headless scraping of protected job boards; slow or geographically distant origins.","solutions":["Run npx playwright install chromium so the primary path works and the fallback is not load-bearing.","In root containers, launch with args: ['--no-sandbox'] or run as a non-root user.","If pages time out at 30s, raise the goto timeout for the slow hosts you care about.","If the fallback's text looks empty, the page is an SPA — fetch the underlying API/JSON the page uses, or capture the JD another way."],"exampleFix":"// before\nbrowser = await chromium.launch({ headless: true });\nawait page.goto(url, { waitUntil: 'domcontentloaded', timeout: 30_000 });\n// after — tolerate root containers and slow origins\nbrowser = await chromium.launch({\n  headless: true,\n  args: process.getuid?.() === 0 ? ['--no-sandbox'] : [],\n});\nawait page.goto(url, { waitUntil: 'domcontentloaded', timeout: 60_000 });","handlingStrategy":"fallback","validationCode":"import { existsSync } from 'node:fs';\nimport { chromium } from 'playwright';\nif (!existsSync(chromium.executablePath())) {\n  throw new Error('Chromium not installed — run: npx playwright install chromium');\n}","typeGuard":null,"tryCatchPattern":"try {\n  return await renderWithPlaywright(url);\n} catch (e) {\n  console.warn(`Playwright failed (${e.message}) — plain-fetch fallback`);\n  return await plainFetch(url); // keep the fallback strictly lesser: no JS, same UA, explicit timeout\n}","preventionTips":["Cache npx playwright install chromium in every CI image","Add --no-sandbox only for root containers on already-isolated hosts","Watch for empty-string returns from the fallback — that is the SPA-shell symptom","Keep an explicit timeout on the fallback fetch; never let it hang after a Playwright timeout"],"tags":["playwright","scraping","fallback","timeout","docker"],"backgroundTag":"playwright-launch-failed","analyzedSha":"60398d6549a46f5266929538af21cfab94badc75","analyzedAt":"2026-08-20T23:00:06.764Z","schemaVersion":2},"datasetVersion":"2026-08-21T18:17:14.833Z"}