santifer/career-ops · error · Error

yourator: unexpected API response on page ${page} — expected

Error message

yourator: unexpected API response on page ${page} — expected { payload: { jobs: [...] } }, got keys: [${json ? Object.keys(json).join(', ') : 'null'}]

What it means

During fetch, each page of the Yourator feed is requested from `${FEED_BASE}?page=N` and the provider expects the documented envelope { payload: { jobs: [...] } }. If json.payload.jobs is missing or not an array, the upstream API contract has changed (or an error/HTML body was returned) and the provider throws rather than emitting garbage. The message includes the page number and the actual top-level keys received for diagnosis.

Source

Thrown at providers/yourator.mjs:189

/** @type {Provider} */
export default {
  id: 'yourator',

  async fetch(entry, ctx) {
    assertYouratorUrl(FEED_BASE);
    // ctx.maxPages is verify-portals.mjs's "first page only" health probe — it
    // always wins over the entry's own bound.
    const maxPages = Math.min(resolveMaxPages(entry), ctx?.maxPages ?? Number.POSITIVE_INFINITY);
    const fallbackCompany = entry?.name;
    const out = [];

    for (let page = 1; page <= maxPages; page++) {
      const url = `${FEED_BASE}?page=${page}`;
      // redirect:'error' prevents SSRF via server-side redirects
      const json = await ctx.fetchJson(url, { redirect: 'error' });
      const jobs = json?.payload?.jobs;
      if (!Array.isArray(jobs)) {
        throw new Error(
          `yourator: unexpected API response on page ${page} — expected { payload: { jobs: [...] } }, got keys: [${json ? Object.keys(json).join(', ') : 'null'}]`,
        );
      }
      for (const j of jobs) {
        const normalized = normalizeYouratorJob(j, fallbackCompany);
        if (normalized) out.push(normalized);
      }
      // `hasMore` is the API's own end-of-board signal and the only stop
      // condition: past the last page it answers with an empty array and
      // hasMore:false. A short-page heuristic is deliberately NOT used — it
      // cannot help (maxPages already bounds a runaway walk) and a single short
      // intermediate page would silently truncate the board.
      if (json.payload.hasMore !== true) break;
      if (page < maxPages) {
        await (ctx.sleep ? ctx.sleep(PAGE_DELAY_MS) : new Promise(r => setTimeout(r, PAGE_DELAY_MS)));
      }
    }
    return out;

View on GitHub (pinned to 1696bec4d0)

Solutions

1. Check the reported keys: if you see error/message/rate-limit keys, retry the scan later or back off — it's a transient upstream response, not a schema change.
2. Verify the current API shape against the live endpoint https://www.yourator.co/api/v4/jobs?page=1 and update the provider's expected envelope (payload.jobs) if Yourator migrated versions.
3. If the schema genuinely changed, update normalizeYouratorJob's input path and the extraction in providers/yourator.mjs to the new field names, then rerun.

Example fix

// before (provider expects v4 envelope)
const jobs = json?.payload?.jobs;

// after (example: API moved to a top-level array)
const jobs = Array.isArray(json?.payload?.jobs) ? json.payload.jobs : Array.isArray(json?.jobs) ? json.jobs : null;

// Probe the feed shape before a full walk:
const probe = await fetch('https://www.yourator.co/api/v4/jobs?page=1').then(r => r.json());
const shapeOk = Array.isArray(probe?.payload?.jobs);
if (!shapeOk) console.warn('Yourator API shape changed; check payload.jobs path', Object.keys(probe ?? {}));

function isJobsEnvelope(json) {
  return json != null
    && typeof json === 'object'
    && json.payload != null
    && typeof json.payload === 'object'
    && Array.isArray(json.payload.jobs);
}

try {
  await scanYouratorAllPages();
} catch (e) {
  if (e.message.startsWith('yourator: unexpected API response')) {
    const page = e.message.match(/page (\d+)/)?.[1];
    console.error(`Yourator schema/availability issue on page ${page}; check for API version change or rate limiting, then retry.`);
  } else throw e;
}

When it happens

Trigger: ctx.fetchJson on page N returns JSON whose payload.jobs is absent or not an array: a v4→v5 API migration that renamed/reshaped payload; a rate-limit or maintenance JSON like { error: '...' }; a captive-portal/proxy returning JSON that isn't the jobs envelope; jobs: null or payload: null on an empty/edge page.

Common situations: Yourator ships a breaking API version change; a CDN/WAF intercepts and returns a JSON error body; transient upstream failure mid-walk on page >1; corporate proxy rewriting responses; following a stale FEED_BASE after a documented endpoint move.

Related errors

• HTTP ${resp.status}: ${t.slice(0, 120)}
• the release pointer is not JSON: ${url}
• glints: unexpected API response — ${JSON.stringify(json).sli
• themuse: unexpected API response on page 0 — expected { resu
• torre: unexpected API response — expected { results: [...] }

AI-assisted analysis of santifer/career-ops@1696bec4d0 (2026-09-01). Data as JSON: /api/errors/9410874c859510a0. Report an issue: GitHub.