santifer/career-ops · error · Error

thehub: unexpected API response on page ${page} — expected {

Error message

thehub: unexpected API response on page ${page} — expected { jobs: { docs: [...] } }, got keys: [${json ? Object.keys(json).join(', ') : 'null'}]

What it means

fetchScope expects each page JSON to have shape { jobs: { docs: [...] } }. If json is null, json.jobs is missing, or json.jobs.docs is not an array, it throws and reports the actual top-level keys. On the first page (state.succeededOnce false) the throw propagates; on later pages the error is caught, logged, and the jobs collected so far are kept — so this error only surfaces when the very first requested page is malformed.

Source

Thrown at providers/thehub.mjs:137

 *
 * @param {string} query the query string beyond `?`, e.g. `countryCode=EU` or `isRemote=true`
 * @param {number} maxPages
 * @param {string | undefined} fallbackCompany
 * @param {Map<string, {title: string, url: string, company: string, location: string}>} byUrl
 * @param {{ fetchJson: (url: string, opts?: object) => Promise<any> }} ctx
 * @param {{ succeededOnce: boolean }} state
 * @returns {Promise<boolean>}
 */
async function fetchScope(query, maxPages, fallbackCompany, byUrl, ctx, state) {
  for (let page = 1; page <= maxPages; page++) {
    const url = `${FEED_BASE}?page=${page}&${query}`;
    let jobs;
    try {
      // redirect:'error' prevents SSRF via server-side redirects
      const json = await ctx.fetchJson(url, { redirect: 'error' });
      jobs = json && json.jobs;
      if (!jobs || !Array.isArray(jobs.docs)) {
        throw new Error(
          `thehub: unexpected API response on page ${page} — expected { jobs: { docs: [...] } }, got keys: [${json ? Object.keys(json).join(', ') : 'null'}]`,
        );
      }
    } catch (err) {
      if (!state.succeededOnce) throw err;
      console.error(`  ⚠ thehub: query "${query}" page ${page} failed (${err.message}) — keeping the ${byUrl.size} jobs collected so far`);
      return false;
    }
    state.succeededOnce = true;
    for (const j of jobs.docs) {
      const normalized = normalizeHubJob(j, fallbackCompany);
      if (normalized && !byUrl.has(normalized.url)) byUrl.set(normalized.url, normalized);
    }
    // Stop at the last page: a short page, or page >= the reported total pages.
    if (jobs.docs.length < PER_PAGE) break;
    if (Number.isInteger(jobs.pages) && page >= jobs.pages) break;
  }
  return true;

View on GitHub (pinned to 9b17a8ac97)

Solutions

  1. Open the FEED_BASE URL (?page=1&<query>) in a browser and compare the actual keys to the error
  2. If the API is temporarily down, retry the scan later
  3. Verify the countryCode / query params in the thehub: block are values the API accepts
  4. If the contract changed, update normalizeHubJob and the docs-array check in providers/thehub.mjs
Defensive patterns

Strategy: try-catch

Type guard

/** @param {unknown} json @returns {json is { jobs: { docs: any[] } }} */
function isHubResponse(json) {
  return !!json && typeof json === 'object'
    && !!json.jobs && typeof json.jobs === 'object'
    && Array.isArray(json.jobs.docs);
}

Try / catch

try {
  const jobs = await provider.fetch(entry, ctx);
} catch (err) {
  if (/thehub: unexpected API response/.test(err.message)) {
    console.warn(`${entry.name}: The Hub API shape changed or is down — skipping`);
    return [];
  }
  throw err;
}

Prevention

When it happens

Trigger: The Hub API (thehub.io/api/v2/jobsandfeatured) returned an error envelope, an HTML page, or a redesigned shape on page 1. A transient outage on the first page also throws. Later-page failures degrade gracefully instead.

Common situations: API contract change after a Hub upgrade, rate-limiting/CAPTCHA on the first request, an invalid countryCode scoping the query to an empty/bad response, or network returning a non-JSON body.

Related errors

AI-assisted analysis of santifer/career-ops@9b17a8ac97 (2026-08-13). Data as JSON: /api/errors/cbeb0a7c4681d0f1. Report an issue: GitHub.