santifer/career-ops · error · Error

joinup: page is missing __NEXT_DATA__ (structure changed?)

Error message

joinup: ${entry.name} page is missing __NEXT_DATA__ (structure changed?)

What it means

The joinup.ch provider throws this when the fetched browse page (https://joinup.ch/browse/jobs) contains no <script id="__NEXT_DATA__"> element at all. joinup.ch server-renders its newest job results into that script; without it there is nothing to parse. Per the code's stated policy, a missing __NEXT_DATA__ is treated as a scraper break, not an empty board, so the provider throws instead of returning zero results.

Solutions

Fetch https://joinup.ch/browse/jobs manually (curl or browser) and check whether <script id="__NEXT_DATA__"> is still present; if it is gone, the site changed framework and providers/joinup.mjs must be rewritten for the new payload (e.g. self.__next_f flight data)
Inspect the fetched HTML for Cloudflare/bot-challenge markers — if blocked, use a browser-based fetcher or rotate user-agent for this provider
Retry later if joinup.ch is briefly down (maintenance page); add retry/backoff around ctx.fetchText
Loosen the regex to match the script regardless of attribute order/extra attributes if the markup changed but the payload remains
Confirm no proxy/CDN in your environment is rewriting or minifying the response HTML

Example fix

// before: exact attribute-form match only
const m = html.match(/<script id="__NEXT_DATA__"[^>]*>([\s\S]*?)<\/script>/);
if (!m) throw new Error(`joinup: ${entry.name} page is missing __NEXT_DATA__ (structure changed?)`);
// after: tolerate attribute order / extra attributes
const m = html.match(/<script[^>]+__NEXT_DATA__[^>]*>([\s\S]*?)<\/script>/)
  || html.match(/<script id="__NEXT_DATA__"[^>]*>([\s\S]*?)<\/script>/);
if (!m) throw new Error(`joinup: ${entry.name} page is missing __NEXT_DATA__ (structure changed?)`);

Defensive patterns

Strategy: validation

Validate before calling

function joinupPageLooksParseable(html) {
  return typeof html === 'string'
    && html.includes('<script id="__NEXT_DATA__"')
    && !/cf-browser-verification|just a moment|attention required/i.test(html);
}
// before trusting fetchText output: if (!joinupPageLooksParseable(html)) retry/flag;

Type guard

function hasNextDataScript(html) {
  return typeof html === 'string'
    && /<script[^>]+__NEXT_DATA__[^>]*>[\s\S]+<\/script>/.test(html);
}

Try / catch

try {
  const hits = fetchJoinupJobs();
} catch (err) {
  if (String(err.message).includes('missing __NEXT_DATA__')) {
    console.error('joinup.ch page had no __NEXT_DATA__ — bot challenge, downtime, or framework migration; retrying later');
    return [];
  }
  throw err;
}

Prevention

Check the fetched HTML for bot-challenge markers (Cloudflare 'Just a moment') whenever this error appears — a 200 status does not mean real content
Watch for joinup.ch migrating from pages-router __NEXT_DATA__ to App Router flight payloads; the regex must be updated then
Pin a fixture of a known-good browse page and diff against live HTML when scans start failing
Retry transient failures with backoff before concluding the site structure changed
Keep redirect:'error' and the pinned https BROWSE_URL so a redirect to an unrelated page cannot masquerade as the board

When it happens

Trigger: Raised in fetch() immediately after ctx.fetchText when the regex /<script id="__NEXT_DATA__"[^>]*>([\s\S]*?)<\/script>/ finds no match: the page is a bot-challenge/CAPTCHA or consent interstitial, the response is an HTML error page, joinup.ch migrated off Next.js pages-router __NEXT_DATA__ (e.g. to the App Router's flight payload), or a redirect/CDN transformed the markup so the id attribute no longer appears verbatim.

Common situations: joinup.ch ships a front-end rewrite (Next.js app router drops the classic __NEXT_DATA__ script); the scanner's user-agent is blocked and it receives a challenge page with HTTP 200; joinup.ch is down and the load balancer returns a maintenance HTML page; the site starts injecting the script with attributes ordered so the exact id="__NEXT_DATA__" pattern still matches but content is async-filled client-side.

Related errors

AI-assisted analysis of santifer/career-ops@aac998c7ed (2026-09-16). Data as JSON: /api/errors/8251eda324c64758. Report an issue: GitHub.

Appendix: source

Thrown at providers/joinup.mjs:52

/** @type {Provider} */
export default {
  id: 'joinup',

  detect(entry) {
    let host;
    try { host = new URL(entry.careers_url || '').hostname; } catch { return null; }
    return /(^|\.)joinup\.ch$/i.test(host) ? { url: BROWSE_URL } : null;
  },

  async fetch(entry, ctx) {
    // redirect:'error' — BROWSE_URL is pinned to joinup.ch (https); a 3xx must
    // not be followed to a private/metadata IP (matches every other provider).
    const html = await ctx.fetchText(BROWSE_URL, { redirect: 'error' });
    const m = html.match(/<script id="__NEXT_DATA__"[^>]*>([\s\S]*?)<\/script>/);
    // Fail closed: a missing/unparseable __NEXT_DATA__ is a scraper break, not an
    // empty board — throw so the scan logs it instead of silently reporting zero.
    if (!m) throw new Error(`joinup: ${entry.name} page is missing __NEXT_DATA__ (structure changed?)`);
    let hits = [];
    try {
      const data = JSON.parse(m[1]);
      const ir = data?.props?.pageProps?.serverState?.initialResults?.jobs?.results;
      hits = Array.isArray(ir) && ir[0]?.hits ? ir[0].hits : [];
    } catch (err) {
      throw new Error(`joinup: ${entry.name} failed to parse __NEXT_DATA__ — ${err.message}`);
    }
    return hits
      .filter(h => h && h.slug && (h.title || h.headline))
      .map(h => ({
        title: h.title || h.headline || '',
        url: `https://joinup.ch/job/${h.slug}`,
        company: h.startup || entry.name || '',
        location: typeof h.location === 'string' ? h.location
          : (h.location?.name || h.location?.city || ''),
        postedAt: toEpochMs(h.created),
      }));

View on GitHub (pinned to aac998c7ed)