What causes "joinup: ${entry.name} failed to parse __NEXT_DATA__ — ${err.message}" in santifer/career-ops?

Log the first ~200 chars of m[1] on failure to see whether the payload is truncated, HTML, or escaped JSON — that distinguishes network truncation from format change

santifer/career-ops · error · Error

joinup: failed to parse __NEXT_DATA__

Error message

joinup: ${entry.name} failed to parse __NEXT_DATA__ — ${err.message}

What it means

The joinup.ch provider found the __NEXT_DATA__ script but JSON.parse of its contents failed, so it rethrows with the underlying parser message. The script body is expected to be a JSON document whose props.pageProps.serverState.initialResults.jobs.results[0].holds the Typesense hits array; malformed JSON means the payload is truncated, HTML-escaped, or otherwise not parseable. As elsewhere, the provider fails closed rather than reporting an empty board.

Solutions

Log the first ~200 chars of m[1] on failure to see whether the payload is truncated, HTML, or escaped JSON — that distinguishes network truncation from format change
Re-fetch and retry: truncation is usually transient, so wrap ctx.fetchText in a retry with backoff for large payloads
If the JSON contains escaped sequences (e.g. \u003C), decode them before JSON.parse or adjust the extraction regex to capture the raw body correctly
Check response integrity (Content-Length vs actual bytes) in ctx.fetchText to detect truncation and treat it as a retryable failure
If joinup.ch changed escaping/format permanently, update the extraction in providers/joinup.mjs to match the new serialization

Example fix

// before: single parse attempt, raw error surfaced
const data = JSON.parse(m[1]);
// after: unescape common HTML-safe sequences and surface payload context
let raw = m[1].replace(/\\u003C/g, '<').replace(/\\u003E/g, '>').replace(/\\u0026/g, '&').replace(/\\"/g, '"');
let data;
try {
  data = JSON.parse(raw);
} catch (err) {
  throw new Error(`joinup: ${entry.name} failed to parse __NEXT_DATA__ — ${err.message}; head: ${raw.slice(0, 120)}`);
}

Defensive patterns

Strategy: retry

Validate before calling

function nextDataBodyLooksLikeJson(raw) {
  if (typeof raw !== 'string') return false;
  const t = raw.trim();
  return t.startsWith('{') && t.endsWith('}');
}
// before JSON.parse: if (!nextDataBodyLooksLikeJson(m[1])) treat as truncated and retry;

Type guard

function parsesAsNextData(raw) {
  try {
    const d = JSON.parse(raw);
    return !!d && typeof d === 'object'
      && Array.isArray(d?.props?.pageProps?.serverState?.initialResults?.jobs?.results);
  } catch { return false; }
}

Try / catch

try {
  const hits = fetchJoinupJobs();
} catch (err) {
  if (String(err.message).startsWith('joinup:') && err.message.includes('failed to parse __NEXT_DATA__')) {
    console.error('joinup __NEXT_DATA__ JSON was malformed (likely truncated) — retrying with backoff');
    return await withRetry(fetchJoinupJobs, 3);
  }
  throw err;
}

Prevention

Retry large hydration payloads — truncation mid-document is the most common cause of JSON.parse failure here
Log a short prefix of the script body on parse failure to distinguish truncation from escaping changes
Handle HTML-safe escaping (\u003C, \u0026) before/inside parsing if joinup.ch changes serialization
Verify Content-Length matches received bytes in the fetch layer to detect and auto-retry truncated responses
Cache a known-good __NEXT_DATA__ fixture to detect format drift quickly when errors spike

When it happens

Trigger: Raised in fetch() inside the try block when JSON.parse(m[1]) throws — the __NEXT_DATA__ body is cut off mid-document (truncated response), contains escaped/entities-mangled JSON, or the response is a challenge/error page that happens to include a __NEXT_DATA__-shaped script with non-JSON content.

Common situations: Network interruption or CDN edge truncates the multi-hundred-KB hydration payload; a security product injects content into the script body; joinup.ch starts HTML-escaping the JSON (e.g. "</script>" escaping changes); an intermediary proxy compresses/transcodes the body in a way the fetcher decodes incorrectly.

Understand the failure class

Background: JSON parse error: "Unexpected token" / "not valid JSON" / "failed to parse" — what JSON parsers are really complaining about — this error's family across 45 libraries.

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of santifer/career-ops@aac998c7ed (2026-09-16). Data as JSON: /api/errors/1e9ac712c7b2100e. Report an issue: GitHub.

Appendix: source

Thrown at providers/joinup.mjs:59

    try { host = new URL(entry.careers_url || '').hostname; } catch { return null; }
    return /(^|\.)joinup\.ch$/i.test(host) ? { url: BROWSE_URL } : null;
  },

  async fetch(entry, ctx) {
    // redirect:'error' — BROWSE_URL is pinned to joinup.ch (https); a 3xx must
    // not be followed to a private/metadata IP (matches every other provider).
    const html = await ctx.fetchText(BROWSE_URL, { redirect: 'error' });
    const m = html.match(/<script id="__NEXT_DATA__"[^>]*>([\s\S]*?)<\/script>/);
    // Fail closed: a missing/unparseable __NEXT_DATA__ is a scraper break, not an
    // empty board — throw so the scan logs it instead of silently reporting zero.
    if (!m) throw new Error(`joinup: ${entry.name} page is missing __NEXT_DATA__ (structure changed?)`);
    let hits = [];
    try {
      const data = JSON.parse(m[1]);
      const ir = data?.props?.pageProps?.serverState?.initialResults?.jobs?.results;
      hits = Array.isArray(ir) && ir[0]?.hits ? ir[0].hits : [];
    } catch (err) {
      throw new Error(`joinup: ${entry.name} failed to parse __NEXT_DATA__ — ${err.message}`);
    }
    return hits
      .filter(h => h && h.slug && (h.title || h.headline))
      .map(h => ({
        title: h.title || h.headline || '',
        url: `https://joinup.ch/job/${h.slug}`,
        company: h.startup || entry.name || '',
        location: typeof h.location === 'string' ? h.location
          : (h.location?.name || h.location?.city || ''),
        postedAt: toEpochMs(h.created),
      }));
  },
};

View on GitHub (pinned to aac998c7ed)