santifer/career-ops · warning · Error

HTTP ${r.status} ${r.statusText}

Error message

HTTP ${r.status} ${r.statusText}

What it means

Thrown by the plain-fetch fallback inside fetchJobPage() when the remote returns a non-2xx response (r.ok is false). It surfaces the raw HTTP status code and reason phrase so the caller can distinguish a dead URL (404) from a block (403) or a server fault (502/503).

Source

Thrown at openrouter-runner.mjs:431

      await page.waitForTimeout(2000); // wait for SPA render
      const text = await page.evaluate(() => {
        document.querySelectorAll('script,style,nav,footer,header').forEach(el => el.remove());
        return (document.body?.innerText || document.body?.textContent || '').replace(/\s+/g, ' ').trim();
      });
      return text.slice(0, 16_000);
    } catch (e) {
      console.warn(`[fetch] Playwright error: ${e.message} — falling back to plain fetch.`);
    } finally {
      if (browser) await browser.close().catch(() => {});
    }
  }

  // Plain HTTP fallback
  try {
    const r = await fetch(url, {
      headers: { 'User-Agent': DEFAULT_USER_AGENT }
    });
    if (!r.ok) throw new Error(`HTTP ${r.status} ${r.statusText}`);
    const html = await r.text();
    return html.replace(/<[^>]+>/g, ' ').replace(/\s+/g, ' ').trim().slice(0, 16_000);
  } catch (e) {
    throw new Error(`Could not fetch job page: ${e.message}`);
  }
}

// ---------------------------------------------------------------------------
// portals.yml parser — reads the canonical schema with js-yaml (same library and
// field names as scan.mjs: `title_filter.positive/negative` + `tracked_companies`),
// so it never drifts from the main scanner. The runner's no-CLI scan path covers
// companies that expose a direct JSON `api:`; careers_url-only / Playwright /
// search-query companies are handled by the full /career-ops scan pipeline.
// `rawOverride` lets tests feed YAML text directly (see test-all.mjs drift guard).
// ---------------------------------------------------------------------------
function normKeywords(v) {
  if (!Array.isArray(v)) return [];
  return v.map(x => String(x ?? '').toLowerCase().trim()).filter(Boolean);

View on GitHub (pinned to 9b17a8ac97)

Solutions

1. Retry once after a short delay — transient 5xx and rate-limit 429 are common.
2. Verify the URL is still live in a browser; if it 404s, mark the pipeline entry dead and drop it.
3. If you see 403/429, prefer the full /career-ops scan pipeline (which uses Playwright with a real browser UA) instead of the runner's plain fetch fallback.
4. Ensure playwright is installed so the Playwright-first path runs before this fallback.

Example fix

// before: plain fetch hit a 403
await fetchJobPage(url);
// after: ensure playwright is installed so the browser path is tried first
npm i playwright && npx playwright install chromium

null

null

try {
  const html = await fetchJobPage(url);
} catch (e) {
  if (/HTTP 4\d\d/.test(e.message)) {
    // dead/block — mark pipeline entry and move on
  } else if (/HTTP 5\d\d|ECONNRESET|ETIMEDOUT/.test(e.message)) {
    // transient — retry with backoff
  } else {
    throw e;
  }
}

When it happens

Trigger: The job posting URL returns 404 (role closed/removed), 403 (bot/WAF block on the default User-Agent), 410 (gone), or 5xx (ATS outage); this fires only after Playwright is unavailable or also failed, since Playwright runs first.

Common situations: A Greenhouse/Lever/Ashby posting was taken down between when pipeline.md was seeded and when the runner fetched it; an ATS rate-limits the runner's IP; a Cloudflare-protected site returns 403 to the plain fetch UA; corporate proxy returns a 502.

Related errors

• Could not fetch job page: ${e.message}
• Refusing non-HTTP(S) URL: ${url}
• Refusing private/loopback host: ${host}
• vc-portfolios: YC API fetch failed — ${err.message}
• vc-portfolios: a16z portfolio fetch failed — ${err.message}

AI-assisted analysis of santifer/career-ops@9b17a8ac97 (2026-08-13). Data as JSON: /api/errors/ad7a128b586e07c0. Report an issue: GitHub.