santifer/career-ops · error · Error

join: __NEXT_DATA__ not found or unexpected structure

Error message

join: __NEXT_DATA__ not found or unexpected structure

What it means

Thrown by join.fetch() on the first page when extractNextData(firstHtml) returns null OR the parsed data lacks props.pageProps.initialState. extractNextData looks for a <script> tag containing __NEXT_DATA__ and JSON.parses its content; it returns null if the tag is absent, the HTML isn't a string, or the JSON is malformed. This is a fail-closed guard: a missing/unexpected Next.js data shape is treated as a scraper break, not an empty board, so it is logged rather than silently reporting zero jobs.

Source

Thrown at providers/join.mjs:62

  detect(entry) {
    return extractSlug(entry.careers_url) ? { url: entry.careers_url } : null;
  },

  async fetch(entry, ctx) {
    const slug = extractSlug(entry.careers_url);
    if (!slug) throw new Error('join: cannot extract slug from careers_url');

    const baseUrl = `https://join.com/companies/${slug}`;
    const allItems = [];

    // redirect:'error' prevents SSRF via server-side redirects; baseUrl is
    // always reconstructed as https://join.com/... so the host is pinned
    // regardless of the original careers_url.
    const firstHtml = await ctx.fetchText(baseUrl, { redirect: 'error' });
    const firstData = extractNextData(firstHtml);
    const state = firstData?.props?.pageProps?.initialState;
    if (!state) throw new Error('join: __NEXT_DATA__ not found or unexpected structure');

    const firstJobs = state.jobs?.items;
    if (!Array.isArray(firstJobs)) throw new Error('join: __NEXT_DATA__ not found or unexpected structure');
    allItems.push(...firstJobs);

    // Honor a context page cap — verify-portals' liveness probe sets
    // `ctx.maxPages: 1` so it only needs to know a board is live, not its
    // full count (mirrors providers/workday.mjs). No effect on real scans,
    // which don't set ctx.maxPages. Kept separate from `maxPages` below so
    // the "raise max_pages" warning only fires when the entry-level cap is
    // what actually truncated the board, not the health-check probe.
    const ctxMaxPages = Number(ctx?.maxPages);
    const ctxCap = ctxMaxPages > 0 ? ctxMaxPages : Infinity;
    const reportedPageCount = state.jobs?.pagination?.pageCount || 0;
    const maxPages = resolveMaxPages(entry);
    const pageCount = Math.min(reportedPageCount, maxPages, ctxCap);
    for (let page = 2; page <= pageCount; page++) {
      const html = await ctx.fetchText(`${baseUrl}?page=${page}`, { redirect: 'error' });

View on GitHub (pinned to 9b17a8ac97)

Solutions

1. Open https://join.com/companies/<slug> in a browser and confirm the board still renders with a __NEXT_DATA__ script tag.
2. If you are being rate-limited, reduce scan frequency or add a delay between requests.
3. If join.com changed its data shape, update extractNextData / the initialState access path in providers/join.mjs.

import { extractNextData } from './providers/join.mjs';
// Probe the board shape before a full scan:
async function joinBoardShapeOk(fetchText, baseUrl) {
  const html = await fetchText(baseUrl, { redirect: 'error' });
  const state = extractNextData(html)?.props?.pageProps?.initialState;
  return Boolean(state);
}

/** @param {any} data @returns {boolean} */
function hasJoinInitialState(data) {
  return Boolean(data && data?.props?.pageProps?.initialState);
}

try {
  jobs = await provider.fetch(entry, ctx);
} catch (err) {
  if (/__NEXT_DATA__ not found/.test(err.message)) {
    console.error(`join ${entry.name}: SSR shape changed or blocked — investigate manually`);
  }
  throw err;
}

When it happens

Trigger: join.com changed its SSR structure (renamed pageProps or initialState keys); the request returned an error page (403/429/500 HTML) instead of the company board; a WAF or bot challenge (Cloudflare interstitial) replaced the real page; the company slug no longer exists and join.com serves a generic page.

Common situations: join.com shipped a frontend change that renamed or restructured the __NEXT_DATA__ payload; rate-limiting returned an HTML challenge page; the company migrated off join.com.

Related errors

• joinup: ${entry.name} page is missing __NEXT_DATA__ (structu
• joinup: ${entry.name} failed to parse __NEXT_DATA__ — ${err.
• API error: ${json.errorMsg || json.errorCode || 'success=fal
• jobvite: could not find companyEId on ${boardUrl} for ${entr
• join: cannot extract slug from careers_url

AI-assisted analysis of santifer/career-ops@9b17a8ac97 (2026-08-13). Data as JSON: /api/errors/7e2e99341d471b5c. Report an issue: GitHub.