{"record":{"id":"6210d5140243ca7a","repo":"santifer/career-ops","slug":"hackernews-unexpected-item-response-for-thread","errorCode":null,"errorMessage":"hackernews: unexpected item response for thread ${threadId}","messagePattern":"hackernews: unexpected item response for thread (.+?)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"providers/hackernews.mjs","lineNumber":154,"sourceCode":"\n/** @type {Provider} */\nexport default {\n  id: 'hackernews',\n\n  async fetch(entry, ctx) {\n    // Step 1: Find the latest \"Who is hiring?\" story id.\n    const searchData = await ctx.fetchJson(SEARCH_URL, { redirect: 'error' });\n    const threadId = resolveLatestThreadId(searchData);\n    if (!threadId) {\n      throw new Error('hackernews: could not find \"Ask HN: Who is hiring?\" thread in search results');\n    }\n\n    const threadHnUrl = `https://news.ycombinator.com/item?id=${threadId}`;\n\n    // Step 2: Fetch the thread item (children = top-level job comments).\n    const item = await ctx.fetchJson(itemUrl(threadId), { redirect: 'error' });\n    if (!item || typeof item !== 'object') {\n      throw new Error(`hackernews: unexpected item response for thread ${threadId}`);\n    }\n\n    const children = /** @type {any} */ (item).children;\n    if (!Array.isArray(children)) return [];\n\n    // Step 3: Parse each comment.\n    const jobs = [];\n    for (const child of children) {\n      // Skip deleted / dead / empty comments.\n      if (!child || child.deleted || child.dead) continue;\n      const text = typeof child.text === 'string' ? child.text : '';\n      if (!text.trim()) continue;\n\n      const parsed = parseHnComment(text, threadHnUrl);\n      if (!parsed) continue;\n\n      jobs.push({\n        title: parsed.title,","sourceCodeStart":136,"sourceCodeEnd":172,"githubUrl":"https://github.com/santifer/career-ops/blob/aac998c7ed7248ea853b720ceeb1fdbeb322fc5d/providers/hackernews.mjs#L136-L172","documentation":"After resolving the thread id, the provider fetches the thread item from the HN (Algolia items) API and requires a JSON object back. If the response is missing or not an object (null, array, string, error payload), fetch() throws this error naming the thread id. Non-object items mean the thread could not be retrieved and children cannot be parsed.","triggerScenarios":"ctx.fetchJson(itemUrl(threadId)) returns null or a non-object: deleted/dead thread id, Algolia item API returning an error JSON or empty body, or a proxy response swallowed as a string.","commonSituations":"Stale cached thread id used after a thread was removed, Algolia item endpoint outage or rate limit, and redirects/blocked requests whose body parses to something other than the expected item object.","solutions":["Open https://news.ycombinator.com/item?id={threadId} to confirm the thread still exists.","Call the item endpoint directly (hn.algolia.com/api/v1/items/{id}) and inspect the raw response.","Retry later if Algolia is rate-limiting or degraded; add a small backoff between requests.","Update the item parsing path in providers/hackernews.mjs if the API shape changed."],"exampleFix":"// before\nconst item = await ctx.fetchJson(itemUrl(threadId), { redirect: 'error' });\n// after (retry once on bad shape)\nlet item = await ctx.fetchJson(itemUrl(threadId), { redirect: 'error' });\nif (!item || typeof item !== 'object') item = await ctx.fetchJson(itemUrl(threadId), { redirect: 'error' });","handlingStrategy":"type-guard","validationCode":"function isItemObject(v) { return v !== null && typeof v === 'object' && !Array.isArray(v); }\nconst item = await (await fetch(itemUrl(id))).json();\nif (!isItemObject(item)) throw new Error(`bad item payload for ${id}`);","typeGuard":"const isHnItem = (v) => v !== null && typeof v === 'object' && !Array.isArray(v) && ('id' in v || 'children' in v);","tryCatchPattern":"try {\n  await provider.fetch(entry, ctx);\n} catch (err) {\n  if (/hackernews: unexpected item response/.test(err.message)) {\n    console.error(`Thread item ${err.message.split('thread ')[1]} unusable — verify it still exists on news.ycombinator.com`);\n    return;\n  }\n  throw err;\n}","preventionTips":["Re-resolve the thread id via search on every run instead of caching a stale id.","Back off and retry on rate-limited or degraded Algolia responses.","Check that fetchJson rejects HTML/error bodies before they reach the object check.","Keep a link to the thread id in logs so it can be checked manually."],"tags":["api","hackernews","unexpected-response","scraper"],"backgroundTag":"unexpected-api-response-shape","analyzedSha":"aac998c7ed7248ea853b720ceeb1fdbeb322fc5d","analyzedAt":"2026-09-16T06:35:29.214Z","contentChangedAt":"2026-09-16T06:35:29.214Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}