{"record":{"id":"8251eda324c64758","repo":"santifer/career-ops","slug":"joinup-entry-name-page-is-missing-next-data","errorCode":null,"errorMessage":"joinup: ${entry.name} page is missing __NEXT_DATA__ (structure changed?)","messagePattern":"joinup: (.+?) page is missing __NEXT_DATA__ \\(structure changed\\?\\)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"providers/joinup.mjs","lineNumber":52,"sourceCode":"\n/** @type {Provider} */\nexport default {\n  id: 'joinup',\n\n  detect(entry) {\n    let host;\n    try { host = new URL(entry.careers_url || '').hostname; } catch { return null; }\n    return /(^|\\.)joinup\\.ch$/i.test(host) ? { url: BROWSE_URL } : null;\n  },\n\n  async fetch(entry, ctx) {\n    // redirect:'error' — BROWSE_URL is pinned to joinup.ch (https); a 3xx must\n    // not be followed to a private/metadata IP (matches every other provider).\n    const html = await ctx.fetchText(BROWSE_URL, { redirect: 'error' });\n    const m = html.match(/<script id=\"__NEXT_DATA__\"[^>]*>([\\s\\S]*?)<\\/script>/);\n    // Fail closed: a missing/unparseable __NEXT_DATA__ is a scraper break, not an\n    // empty board — throw so the scan logs it instead of silently reporting zero.\n    if (!m) throw new Error(`joinup: ${entry.name} page is missing __NEXT_DATA__ (structure changed?)`);\n    let hits = [];\n    try {\n      const data = JSON.parse(m[1]);\n      const ir = data?.props?.pageProps?.serverState?.initialResults?.jobs?.results;\n      hits = Array.isArray(ir) && ir[0]?.hits ? ir[0].hits : [];\n    } catch (err) {\n      throw new Error(`joinup: ${entry.name} failed to parse __NEXT_DATA__ — ${err.message}`);\n    }\n    return hits\n      .filter(h => h && h.slug && (h.title || h.headline))\n      .map(h => ({\n        title: h.title || h.headline || '',\n        url: `https://joinup.ch/job/${h.slug}`,\n        company: h.startup || entry.name || '',\n        location: typeof h.location === 'string' ? h.location\n          : (h.location?.name || h.location?.city || ''),\n        postedAt: toEpochMs(h.created),\n      }));","sourceCodeStart":34,"sourceCodeEnd":70,"githubUrl":"https://github.com/santifer/career-ops/blob/aac998c7ed7248ea853b720ceeb1fdbeb322fc5d/providers/joinup.mjs#L34-L70","documentation":"The joinup.ch provider throws this when the fetched browse page (https://joinup.ch/browse/jobs) contains no <script id=\"__NEXT_DATA__\"> element at all. joinup.ch server-renders its newest job results into that script; without it there is nothing to parse. Per the code's stated policy, a missing __NEXT_DATA__ is treated as a scraper break, not an empty board, so the provider throws instead of returning zero results.","triggerScenarios":"Raised in fetch() immediately after ctx.fetchText when the regex /<script id=\"__NEXT_DATA__\"[^>]*>([\\s\\S]*?)<\\/script>/ finds no match: the page is a bot-challenge/CAPTCHA or consent interstitial, the response is an HTML error page, joinup.ch migrated off Next.js pages-router __NEXT_DATA__ (e.g. to the App Router's flight payload), or a redirect/CDN transformed the markup so the id attribute no longer appears verbatim.","commonSituations":"joinup.ch ships a front-end rewrite (Next.js app router drops the classic __NEXT_DATA__ script); the scanner's user-agent is blocked and it receives a challenge page with HTTP 200; joinup.ch is down and the load balancer returns a maintenance HTML page; the site starts injecting the script with attributes ordered so the exact id=\"__NEXT_DATA__\" pattern still matches but content is async-filled client-side.","solutions":["Fetch https://joinup.ch/browse/jobs manually (curl or browser) and check whether <script id=\"__NEXT_DATA__\"> is still present; if it is gone, the site changed framework and providers/joinup.mjs must be rewritten for the new payload (e.g. self.__next_f flight data)","Inspect the fetched HTML for Cloudflare/bot-challenge markers — if blocked, use a browser-based fetcher or rotate user-agent for this provider","Retry later if joinup.ch is briefly down (maintenance page); add retry/backoff around ctx.fetchText","Loosen the regex to match the script regardless of attribute order/extra attributes if the markup changed but the payload remains","Confirm no proxy/CDN in your environment is rewriting or minifying the response HTML"],"exampleFix":"// before: exact attribute-form match only\nconst m = html.match(/<script id=\"__NEXT_DATA__\"[^>]*>([\\s\\S]*?)<\\/script>/);\nif (!m) throw new Error(`joinup: ${entry.name} page is missing __NEXT_DATA__ (structure changed?)`);\n// after: tolerate attribute order / extra attributes\nconst m = html.match(/<script[^>]+__NEXT_DATA__[^>]*>([\\s\\S]*?)<\\/script>/)\n  || html.match(/<script id=\"__NEXT_DATA__\"[^>]*>([\\s\\S]*?)<\\/script>/);\nif (!m) throw new Error(`joinup: ${entry.name} page is missing __NEXT_DATA__ (structure changed?)`);","handlingStrategy":"validation","validationCode":"function joinupPageLooksParseable(html) {\n  return typeof html === 'string'\n    && html.includes('<script id=\"__NEXT_DATA__\"')\n    && !/cf-browser-verification|just a moment|attention required/i.test(html);\n}\n// before trusting fetchText output: if (!joinupPageLooksParseable(html)) retry/flag;","typeGuard":"function hasNextDataScript(html) {\n  return typeof html === 'string'\n    && /<script[^>]+__NEXT_DATA__[^>]*>[\\s\\S]+<\\/script>/.test(html);\n}","tryCatchPattern":"try {\n  const hits = fetchJoinupJobs();\n} catch (err) {\n  if (String(err.message).includes('missing __NEXT_DATA__')) {\n    console.error('joinup.ch page had no __NEXT_DATA__ — bot challenge, downtime, or framework migration; retrying later');\n    return [];\n  }\n  throw err;\n}","preventionTips":["Check the fetched HTML for bot-challenge markers (Cloudflare 'Just a moment') whenever this error appears — a 200 status does not mean real content","Watch for joinup.ch migrating from pages-router __NEXT_DATA__ to App Router flight payloads; the regex must be updated then","Pin a fixture of a known-good browse page and diff against live HTML when scans start failing","Retry transient failures with backoff before concluding the site structure changed","Keep redirect:'error' and the pinned https BROWSE_URL so a redirect to an unrelated page cannot masquerade as the board"],"tags":["scraper","nextjs","bot-protection","html-parsing"],"backgroundTag":"unexpected-response-shape","analyzedSha":"aac998c7ed7248ea853b720ceeb1fdbeb322fc5d","analyzedAt":"2026-09-16T06:35:29.214Z","contentChangedAt":"2026-09-16T06:35:29.214Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}