{"record":{"id":"44ec7bf0495423e0","repo":"santifer/career-ops","slug":"getro-label-could-not-resolve-collection-id","errorCode":null,"errorMessage":"getro: ${label} — could not resolve collection_id from ${careersUrl.href} (no network.id found in __NEXT_DATA__; page structure may have changed — set getro_collection: N on this entry as a fallback)","messagePattern":"getro: (.+?) — could not resolve collection_id from (.+?) \\(no network\\.id found in __NEXT_DATA__; page structure may have changed — set getro_collection: N on this entry as a fallback\\)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"providers/getro.mjs","lineNumber":157,"sourceCode":"  return null;\n}\n\n/** Override wins; otherwise fetch careers_url and parse __NEXT_DATA__. */\nasync function resolveCollectionId(entry, ctx, careersUrl) {\n  const override = resolveCollectionOverride(entry);\n  if (override) return override;\n\n  const label = entry?.name || careersUrl.href;\n  // Retried like every page fetch below — this single request runs BEFORE\n  // pagination even starts, so without a retry a transient blip here (DNS/TLS/\n  // connection reset) fails the whole board before a single page is fetched.\n  const html = await fetchTextWithRetry(ctx, careersUrl.href, {\n    redirect: 'error',\n    headers: { accept: 'text/html', 'user-agent': BROWSER_LIKE_USER_AGENT },\n  });\n  const id = extractCollectionId(html);\n  if (!id) {\n    throw new Error(\n      `getro: ${label} — could not resolve collection_id from ${careersUrl.href} (no network.id found in ` +\n      `__NEXT_DATA__; page structure may have changed — set getro_collection: N on this entry as a fallback)`,\n    );\n  }\n  return id;\n}\n\n/**\n * `{min, max, currency}` shape scan.mjs's salary_filter consumes, or null\n * when there's no usable figure. A non-year compensation_period\n * (hourly/monthly/etc.) is treated as \"no usable annual figure\".\n */\nfunction getroSalary(job) {\n  const period = typeof job?.compensation_period === 'string' ? job.compensation_period.trim().toLowerCase() : '';\n  if (period && period !== 'year') return null;\n  const minCents = Number(job?.compensation_amount_min_cents);\n  const maxCents = Number(job?.compensation_amount_max_cents);\n  const min = Number.isFinite(minCents) && minCents > 0 ? minCents / 100 : null;","sourceCodeStart":139,"sourceCodeEnd":175,"githubUrl":"https://github.com/santifer/career-ops/blob/1696bec4d021768e7359f9aad6b329cba883da20/providers/getro.mjs#L139-L175","documentation":"The getro provider auto-discovers a collection_id by fetching the company's careers page (with redirect:'error') and scraping network.id out of the embedded __NEXT_DATA__ JSON. If extractCollectionId finds no id in the HTML, resolveCollectionId throws this error naming the entry label and the careers URL. It means the page no longer exposes the id where expected — or the page wasn't reached as anticipated.","triggerScenarios":"fetchTextWithRetry returns HTML without a network.id inside __NEXT_DATA__ — e.g. a client-side-only rendered page, a bot-challenge/Cloudflare interstitial, a redesigned careers page, or the company moved off Getro — for a getro entry with no getro_collection override.","commonSituations":"Getro changed its Next.js data structure; the company's careers URL now redirects (blocked by redirect:'error'); scraping is blocked by a WAF returning challenge HTML; the entry's careers_url points to the wrong page.","solutions":["Set the documented fallback on the entry: `getro_collection: N` with the collection id taken from the careers page's __NEXT_DATA__ (view-source in a browser) or an existing API call.","Open careersUrl.href in a browser and confirm it still renders a Getro board; update careers_url if the company moved.","Inspect the page source to see if __NEXT_DATA__ moved to a new script/structure and update extractCollectionId.","Check whether a bot challenge is returned (curl the URL); reduce request frequency or adjust headers if so."],"exampleFix":"// before (portals entry)\n{ name: 'Acme', provider: 'getro', careers_url: 'https://acme.getro.com/careers' }\n// after\n{ name: 'Acme', provider: 'getro', careers_url: 'https://acme.getro.com/careers', getro_collection: 12345 }","handlingStrategy":"fallback","validationCode":"// Pre-check the page before auto-discovery:\nconst html = await fetch(careersUrl).then(r => r.text());\nconst hasId = /\"network\"\\s*:\\s*\\{[^}]*\"id\"\\s*:\\s*\\d+/.test(html);\nif (!hasId) console.warn(`getro: no network.id on ${careersUrl} — set getro_collection manually`);","typeGuard":"function hasResolvedCollection(entry) {\n  return Number.isInteger(entry.getro_collection) && entry.getro_collection > 0;\n}","tryCatchPattern":"try {\n  const id = await resolveCollectionId(entry, ctx, careersUrl);\n} catch (e) {\n  if (e.message.includes('could not resolve collection_id')) {\n    console.warn(`${entry.name}: ${e.message}`);\n    if (entry.getro_collection) return useOverride(entry.getro_collection);\n    return null; // skip entry, keep scan running\n  }\n  throw e;\n}","preventionTips":["Always pin getro_collection on entries you care about instead of relying on scraping.","Re-verify careers pages after Getro UI updates; note the id from view-source __NEXT_DATA__.","Keep provider scrape logic resilient: test extractCollectionId against saved page snapshots.","Skip-and-report rather than crash: collect failed entries for manual follow-up."],"tags":["scraping","getro","config","page-structure-change"],"backgroundTag":"selector-not-found","analyzedSha":"1696bec4d021768e7359f9aad6b329cba883da20","analyzedAt":"2026-09-01T19:19:23.111Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}