What causes "join: __NEXT_DATA__ not found or unexpected structure" in santifer/career-ops?

Fetch the board URL manually and confirm it is a real careers page; if it shows a bot check or 404, the slug is wrong or the board is gone — fix careers_url or remove the entry.

santifer/career-ops · error · Error

join: __NEXT_DATA__ not found or unexpected structure

Error message

join: __NEXT_DATA__ not found or unexpected structure

What it means

join.com renders jobs server-side with Next.js; the provider parses the embedded <script id="__NEXT_DATA__"> JSON via extractNextData() and reads props.pageProps.initialState. It throws this error when the script tag is missing entirely, the JSON fails to parse, or initialState is absent — i.e. the response is not a valid join.com company board page.

Solutions

Fetch the board URL manually and confirm it is a real careers page; if it shows a bot check or 404, the slug is wrong or the board is gone — fix careers_url or remove the entry.
Retry later if a transient bot-protection or CDN issue served non-board HTML.
If join.com changed its page structure, update extractNextData()/the initialState path in providers/join.mjs to match the new __NEXT_DATA__ layout.

Example fix

// before (portals.yml)
careers_url: https://join.com/companies/acme-gone
// after
careers_url: https://join.com/companies/acme

Defensive patterns

Strategy: try-catch

Validate before calling

const html = await fetch(boardUrl).then(r => r.text());
if (!html.includes('__NEXT_DATA__')) console.warn(`${boardUrl} did not serve a Next.js board page`);

Try / catch

try {
  await provider.fetch(entry, ctx);
} catch (e) {
  if (e.message.includes('__NEXT_DATA__ not found')) {
    // bot-check page, dead slug, or join.com markup change — verify board manually
  }
}

Prevention

Verify the board opens in a plain browser (no interstitial) before adding it to portals.yml.
Remove entries for companies that left join.com promptly.
Pin a smoke test that scans one known-good join.com board to catch framework changes early.

When it happens

Trigger: ctx.fetchText(baseUrl, {redirect:'error'}) returned HTML without __NEXT_DATA__ (e.g. a Cloudflare/human-check page, a 404 page for a dead slug, a maintenance page), or the JSON parsed but initialState is null/missing.

Common situations: Bot-protection interstitial served instead of the page; company closed its join.com board; join.com deployed a framework change (e.g. moving off Next.js or renaming initialState) breaking the extractor.

Related errors

AI-assisted analysis of santifer/career-ops@aac998c7ed (2026-09-16). Data as JSON: /api/errors/7e2e99341d471b5c. Report an issue: GitHub.

Appendix: source

Thrown at providers/join.mjs:62

  detect(entry) {
    return extractSlug(entry.careers_url) ? { url: entry.careers_url } : null;
  },

  async fetch(entry, ctx) {
    const slug = extractSlug(entry.careers_url);
    if (!slug) throw new Error('join: cannot extract slug from careers_url');

    const baseUrl = `https://join.com/companies/${slug}`;
    const allItems = [];

    // redirect:'error' prevents SSRF via server-side redirects; baseUrl is
    // always reconstructed as https://join.com/... so the host is pinned
    // regardless of the original careers_url.
    const firstHtml = await ctx.fetchText(baseUrl, { redirect: 'error' });
    const firstData = extractNextData(firstHtml);
    const state = firstData?.props?.pageProps?.initialState;
    if (!state) throw new Error('join: __NEXT_DATA__ not found or unexpected structure');

    const firstJobs = state.jobs?.items;
    if (!Array.isArray(firstJobs)) throw new Error('join: __NEXT_DATA__ not found or unexpected structure');
    allItems.push(...firstJobs);

    // Honor a context page cap — verify-portals' liveness probe sets
    // `ctx.maxPages: 1` so it only needs to know a board is live, not its
    // full count (mirrors providers/workday.mjs). No effect on real scans,
    // which don't set ctx.maxPages. Kept separate from `maxPages` below so
    // the "raise max_pages" warning only fires when the entry-level cap is
    // what actually truncated the board, not the health-check probe.
    const ctxMaxPages = Number(ctx?.maxPages);
    const ctxCap = ctxMaxPages > 0 ? ctxMaxPages : Infinity;
    const reportedPageCount = state.jobs?.pagination?.pageCount || 0;
    const maxPages = resolveMaxPages(entry);
    const pageCount = Math.min(reportedPageCount, maxPages, ctxCap);
    for (let page = 2; page <= pageCount; page++) {
      const html = await ctx.fetchText(`${baseUrl}?page=${page}`, { redirect: 'error' });

View on GitHub (pinned to aac998c7ed)