{"record":{"id":"cbeb0a7c4681d0f1","repo":"santifer/career-ops","slug":"thehub-unexpected-api-response-on-page-page","errorCode":null,"errorMessage":"thehub: unexpected API response on page ${page} — expected { jobs: { docs: [...] } }, got keys: [${json ? Object.keys(json).join(', ') : 'null'}]","messagePattern":"thehub: unexpected API response on page (.+?) — expected (.+?) \\}, got keys: \\[(.+?)\\]","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"providers/thehub.mjs","lineNumber":137,"sourceCode":" *\n * @param {string} query the query string beyond `?`, e.g. `countryCode=EU` or `isRemote=true`\n * @param {number} maxPages\n * @param {string | undefined} fallbackCompany\n * @param {Map<string, {title: string, url: string, company: string, location: string}>} byUrl\n * @param {{ fetchJson: (url: string, opts?: object) => Promise<any> }} ctx\n * @param {{ succeededOnce: boolean }} state\n * @returns {Promise<boolean>}\n */\nasync function fetchScope(query, maxPages, fallbackCompany, byUrl, ctx, state) {\n  for (let page = 1; page <= maxPages; page++) {\n    const url = `${FEED_BASE}?page=${page}&${query}`;\n    let jobs;\n    try {\n      // redirect:'error' prevents SSRF via server-side redirects\n      const json = await ctx.fetchJson(url, { redirect: 'error' });\n      jobs = json && json.jobs;\n      if (!jobs || !Array.isArray(jobs.docs)) {\n        throw new Error(\n          `thehub: unexpected API response on page ${page} — expected { jobs: { docs: [...] } }, got keys: [${json ? Object.keys(json).join(', ') : 'null'}]`,\n        );\n      }\n    } catch (err) {\n      if (!state.succeededOnce) throw err;\n      console.error(`  ⚠ thehub: query \"${query}\" page ${page} failed (${err.message}) — keeping the ${byUrl.size} jobs collected so far`);\n      return false;\n    }\n    state.succeededOnce = true;\n    for (const j of jobs.docs) {\n      const normalized = normalizeHubJob(j, fallbackCompany);\n      if (normalized && !byUrl.has(normalized.url)) byUrl.set(normalized.url, normalized);\n    }\n    // Stop at the last page: a short page, or page >= the reported total pages.\n    if (jobs.docs.length < PER_PAGE) break;\n    if (Number.isInteger(jobs.pages) && page >= jobs.pages) break;\n  }\n  return true;","sourceCodeStart":119,"sourceCodeEnd":155,"githubUrl":"https://github.com/santifer/career-ops/blob/9b17a8ac97b398a496b38e423ae24e433b43254f/providers/thehub.mjs#L119-L155","documentation":"fetchScope expects each page JSON to have shape { jobs: { docs: [...] } }. If json is null, json.jobs is missing, or json.jobs.docs is not an array, it throws and reports the actual top-level keys. On the first page (state.succeededOnce false) the throw propagates; on later pages the error is caught, logged, and the jobs collected so far are kept — so this error only surfaces when the very first requested page is malformed.","triggerScenarios":"The Hub API (thehub.io/api/v2/jobsandfeatured) returned an error envelope, an HTML page, or a redesigned shape on page 1. A transient outage on the first page also throws. Later-page failures degrade gracefully instead.","commonSituations":"API contract change after a Hub upgrade, rate-limiting/CAPTCHA on the first request, an invalid countryCode scoping the query to an empty/bad response, or network returning a non-JSON body.","solutions":["Open the FEED_BASE URL (?page=1&<query>) in a browser and compare the actual keys to the error","If the API is temporarily down, retry the scan later","Verify the countryCode / query params in the thehub: block are values the API accepts","If the contract changed, update normalizeHubJob and the docs-array check in providers/thehub.mjs"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":"/** @param {unknown} json @returns {json is { jobs: { docs: any[] } }} */\nfunction isHubResponse(json) {\n  return !!json && typeof json === 'object'\n    && !!json.jobs && typeof json.jobs === 'object'\n    && Array.isArray(json.jobs.docs);\n}","tryCatchPattern":"try {\n  const jobs = await provider.fetch(entry, ctx);\n} catch (err) {\n  if (/thehub: unexpected API response/.test(err.message)) {\n    console.warn(`${entry.name}: The Hub API shape changed or is down — skipping`);\n    return [];\n  }\n  throw err;\n}","preventionTips":["Rely on the built-in partial-success behavior: later-page failures already degrade gracefully.","Monitor the The Hub API contract after version bumps.","Validate countryCode / query params against what the API accepts."],"tags":["api-contract","runtime","thehub","response-shape"],"backgroundTag":null,"analyzedSha":"9b17a8ac97b398a496b38e423ae24e433b43254f","analyzedAt":"2026-08-13T00:48:39.135Z","schemaVersion":2},"datasetVersion":"2026-08-13T04:17:16.726Z"}