{"record":{"id":"426f9fe62cc70311","repo":"can1357/oh-my-pi","slug":"failed-to-parse-best-stories","errorCode":null,"errorMessage":"Failed to parse best stories","messagePattern":"Failed to parse best stories","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"packages/coding-agent/src/web/scrapers/hackernews.ts","lineNumber":171,"sourceCode":"\t\t} else if (parsed.pathname === \"/\" || parsed.pathname === \"/news\") {\n\t\t\tconst { content: raw, ok } = await loadPage(`${API_BASE}/topstories.json`, { timeout, signal });\n\t\t\tif (!ok) throw new Error(\"Failed to fetch top stories\");\n\t\t\tconst ids = tryParseJson<number[]>(raw);\n\t\t\tif (!ids) throw new Error(\"Failed to parse top stories\");\n\t\t\tcontent = await renderListing(ids, timeout, \"Hacker News - Top Stories\", signal);\n\t\t\tnotes.push(\"Fetched top 20 stories from HN front page\");\n\t\t} else if (parsed.pathname === \"/newest\") {\n\t\t\tconst { content: raw, ok } = await loadPage(`${API_BASE}/newstories.json`, { timeout, signal });\n\t\t\tif (!ok) throw new Error(\"Failed to fetch new stories\");\n\t\t\tconst ids = tryParseJson<number[]>(raw);\n\t\t\tif (!ids) throw new Error(\"Failed to parse new stories\");\n\t\t\tcontent = await renderListing(ids, timeout, \"Hacker News - New Stories\", signal);\n\t\t\tnotes.push(\"Fetched top 20 new stories\");\n\t\t} else if (parsed.pathname === \"/best\") {\n\t\t\tconst { content: raw, ok } = await loadPage(`${API_BASE}/beststories.json`, { timeout, signal });\n\t\t\tif (!ok) throw new Error(\"Failed to fetch best stories\");\n\t\t\tconst ids = tryParseJson<number[]>(raw);\n\t\t\tif (!ids) throw new Error(\"Failed to parse best stories\");\n\t\t\tcontent = await renderListing(ids, timeout, \"Hacker News - Best Stories\", signal);\n\t\t\tnotes.push(\"Fetched top 20 best stories\");\n\t\t} else {\n\t\t\treturn null;\n\t\t}\n\n\t\treturn buildResult(content, { url, method: \"hackernews\", fetchedAt, notes });\n\t} catch (err) {\n\t\tconst errorMsg = err instanceof Error ? err.message : String(err);\n\t\tnotes.push(`Error: ${errorMsg}`);\n\t\treturn buildResult(`# Error fetching Hacker News content\\n\\n${errorMsg}`, {\n\t\t\turl,\n\t\t\tmethod: \"hackernews\",\n\t\t\tfetchedAt,\n\t\t\tnotes,\n\t\t});\n\t}\n};","sourceCodeStart":153,"sourceCodeEnd":189,"githubUrl":"https://github.com/can1357/oh-my-pi/blob/969062200754ea02cfac922e5ebb8c608c079e15/packages/coding-agent/src/web/scrapers/hackernews.ts#L153-L189","documentation":"Thrown when the body fetched from `beststories.json` cannot be parsed as a JSON number array by `tryParseJson<number[]>` while handling /best. The fetch reported ok:true but the decoded content is not the expected JSON — typically an injected HTML page or truncated payload.","triggerScenarios":"GET to https://hacker-news.firebaseio.com/v0/beststories.json returned ok:true but body is not valid JSON — challenge page, captive portal, empty body, or truncation at maxBytes mid-array.","commonSituations":"Corporate proxy returning 200 HTML; Cloudflare/CDN interstitial; truncated body; charset mismatch garbling bytes.","solutions":["Inspect the raw body in diagnostics to identify what was actually returned.","Check for proxies/captive portals intercepting HTTPS with 200 responses.","Retry; transient truncation is most common.","Fall back to Algolia HN API.","Confirm no middleware strips or rewrites the response body."],"exampleFix":"// before\nconst ids = tryParseJson<number[]>(raw);\nif (!ids) throw new Error(\"Failed to parse best stories\");\n// after\nconst ids = tryParseJson<number[]>(raw);\nif (!ids) throw new Error(`beststories.json non-JSON body (${raw.length} bytes), head: ${raw.slice(0, 120)}`);","handlingStrategy":"validation","validationCode":"const ids = tryParseJson<number[]>(raw);\nif (!Array.isArray(ids) || ids.some(id => typeof id !== \"number\")) {\n  throw new Error(`unexpected beststories payload: ${String(raw).slice(0, 120)}`);\n}","typeGuard":"function isIdArray(v: unknown): v is number[] {\n  return Array.isArray(v) && v.every(id => typeof id === \"number\");\n}","tryCatchPattern":"try {\n  const ids = tryParseJson<number[]>(raw);\n  if (!ids) throw new Error(`non-JSON body: ${raw.slice(0, 120)}`);\n} catch (err) {\n  logger.warn(\"beststories.json body not JSON — inspect for proxy/challenge page\", { head: raw.slice(0, 200) });\n}","preventionTips":["Log the body head on parse failure to identify HTML injection.","Check proxy middleware that rewrites response bodies.","Keep identity content-encoding to avoid decode corruption.","Compare content-length vs received bytes to catch truncation.","Fall back to Algolia if Firebase keeps returning malformed bodies."],"tags":["json","parsing","hacker-news","proxy"],"backgroundTag":"invalid-json-response","analyzedSha":"969062200754ea02cfac922e5ebb8c608c079e15","analyzedAt":"2026-08-31T10:29:35.737Z","schemaVersion":2},"datasetVersion":"2026-08-31T14:17:45.589Z"}