{"record":{"id":"a724dcbb4f12ba3d","repo":"can1357/oh-my-pi","slug":"failed-to-fetch-top-stories","errorCode":null,"errorMessage":"Failed to fetch top stories","messagePattern":"Failed to fetch top stories","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"packages/coding-agent/src/web/scrapers/hackernews.ts","lineNumber":155,"sourceCode":"\tconst parsed = new URL(url);\n\tif (!parsed.hostname.includes(\"news.ycombinator.com\")) return null;\n\n\tconst notes: string[] = [];\n\tlet content = \"\";\n\tconst fetchedAt = new Date().toISOString();\n\n\ttry {\n\t\tconst itemId = parsed.searchParams.get(\"id\");\n\n\t\tif (itemId) {\n\t\t\tconst item = await fetchItem(parseInt(itemId, 10), timeout, signal);\n\t\t\tif (!item) throw new Error(`Failed to fetch item ${itemId}`);\n\n\t\t\tcontent = await renderStory(item, timeout, 0, signal);\n\t\t\tnotes.push(`Fetched HN item ${itemId} with top-level comments (depth 2)`);\n\t\t} else if (parsed.pathname === \"/\" || parsed.pathname === \"/news\") {\n\t\t\tconst { content: raw, ok } = await loadPage(`${API_BASE}/topstories.json`, { timeout, signal });\n\t\t\tif (!ok) throw new Error(\"Failed to fetch top stories\");\n\t\t\tconst ids = tryParseJson<number[]>(raw);\n\t\t\tif (!ids) throw new Error(\"Failed to parse top stories\");\n\t\t\tcontent = await renderListing(ids, timeout, \"Hacker News - Top Stories\", signal);\n\t\t\tnotes.push(\"Fetched top 20 stories from HN front page\");\n\t\t} else if (parsed.pathname === \"/newest\") {\n\t\t\tconst { content: raw, ok } = await loadPage(`${API_BASE}/newstories.json`, { timeout, signal });\n\t\t\tif (!ok) throw new Error(\"Failed to fetch new stories\");\n\t\t\tconst ids = tryParseJson<number[]>(raw);\n\t\t\tif (!ids) throw new Error(\"Failed to parse new stories\");\n\t\t\tcontent = await renderListing(ids, timeout, \"Hacker News - New Stories\", signal);\n\t\t\tnotes.push(\"Fetched top 20 new stories\");\n\t\t} else if (parsed.pathname === \"/best\") {\n\t\t\tconst { content: raw, ok } = await loadPage(`${API_BASE}/beststories.json`, { timeout, signal });\n\t\t\tif (!ok) throw new Error(\"Failed to fetch best stories\");\n\t\t\tconst ids = tryParseJson<number[]>(raw);\n\t\t\tif (!ids) throw new Error(\"Failed to parse best stories\");\n\t\t\tcontent = await renderListing(ids, timeout, \"Hacker News - Best Stories\", signal);\n\t\t\tnotes.push(\"Fetched top 20 best stories\");","sourceCodeStart":137,"sourceCodeEnd":173,"githubUrl":"https://github.com/can1357/oh-my-pi/blob/969062200754ea02cfac922e5ebb8c608c079e15/packages/coding-agent/src/web/scrapers/hackernews.ts#L137-L173","documentation":"Thrown by handleHackerNews when `loadPage` returns `ok: false` for the Firebase `topstories.json` endpoint while handling the HN front page (`/` or `/news`). loadPage reports ok:false on transport errors, non-2xx status, or when the response has no readable body after trying its user-agent rotation (curl, TextBot, Chrome UA).","triggerScenarios":"Calling web-fetch on https://news.ycombinator.com/ or /news when the GET to https://hacker-news.firebaseio.com/v0/topstories.json fails: network outage, DNS failure, Firebase 4xx/5xx, or all three user-agent attempts blocked.","commonSituations":"HN Firebase API outage or maintenance; sandboxed/air-gapped environment without internet; DNS resolution failure for hacker-news.firebaseio.com; rate limiting by Google's Firebase hosting; IPv6-only breakage.","solutions":["Confirm the machine can reach hacker-news.firebaseio.com (curl -I https://hacker-news.firebaseio.com/v0/topstories.json) and retry.","Check the result block's notes/error for the loadPage transport error detail to identify DNS vs HTTP vs timeout.","If Firebase is unavailable, fall back to the Algolia HN API (https://hn.algolia.com/api/v1/search?tags=front_page).","Increase the tool timeout if requests were timing out.","Check for proxy/firewall rules blocking requests to Google Firebase hosting domains."],"exampleFix":"// before: blind retry loop\nfor (let i = 0; i < 5; i++) await webFetch(\"https://news.ycombinator.com/\");\n// after: probe the upstream API first, then degrade gracefully\nconst probe = await fetch(\"https://hacker-news.firebaseio.com/v0/topstories.json\");\nif (!probe.ok) {\n  const alt = await fetch(\"https://hn.algolia.com/api/v1/search?tags=front_page\");\n  // use Algolia results instead\n}","handlingStrategy":"retry","validationCode":"const probe = await fetch(\"https://hacker-news.firebaseio.com/v0/topstories.json\", { method: \"HEAD\" });\nif (!probe.ok) throw new Error(`HN Firebase unreachable: ${probe.status}`);","typeGuard":null,"tryCatchPattern":"try {\n  const res = await webFetch(\"https://news.ycombinator.com/\");\n} catch (err) {\n  if (err instanceof ToolAbortError) throw err;\n  const alt = await fetch(\"https://hn.algolia.com/api/v1/search?tags=front_page\");\n  // degrade to Algolia listing\n}","preventionTips":["HEAD-probe the Firebase endpoint before rendering the front page.","Cache the story-id list briefly to reduce redundant upstream calls.","Check connectivity/DNS in sandboxed environments before web-fetch calls.","Keep an Algolia API fallback wired in.","Watch for Firebase 429s and back off rather than hammering."],"tags":["network","http","hacker-news","firebase"],"backgroundTag":"fetch-failed-non-ok-response","analyzedSha":"969062200754ea02cfac922e5ebb8c608c079e15","analyzedAt":"2026-08-31T10:29:35.737Z","schemaVersion":2},"datasetVersion":"2026-08-31T14:17:45.589Z"}