{"record":{"id":"017eec6696bcc8b8","repo":"jackwener/OpenCLI","slug":"lobsters-domain-returned-malformed-json-err-me","errorCode":null,"errorMessage":"lobsters domain returned malformed JSON: ${err?.message ?? err}","messagePattern":"lobsters domain returned malformed JSON: (.+?)","errorType":"exception","errorClass":"CommandExecutionError","httpStatus":null,"severity":"error","filePath":"clis/lobsters/domain.js","lineNumber":73,"sourceCode":"        }\n        catch (err) {\n            throw new CommandExecutionError(\n                `lobsters domain request failed: ${err?.message ?? err}`,\n                'Check that lobste.rs is reachable from this network.',\n            );\n        }\n        if (resp.status === 404) {\n            throw new EmptyResultError('lobsters domain', `No Lobste.rs stories found for domain \"${domain}\".`);\n        }\n        if (!resp.ok) {\n            throw new CommandExecutionError(`lobsters domain returned HTTP ${resp.status}`);\n        }\n        let body;\n        try {\n            body = await resp.json();\n        }\n        catch (err) {\n            throw new CommandExecutionError(`lobsters domain returned malformed JSON: ${err?.message ?? err}`);\n        }\n        const list = Array.isArray(body) ? body : [];\n        if (!list.length) {\n            throw new EmptyResultError('lobsters domain', `No Lobste.rs stories found for domain \"${domain}\".`);\n        }\n        return list.slice(0, limit).map((item, i) => ({\n            rank: i + 1,\n            id: String(item.short_id ?? ''),\n            title: String(item.title ?? ''),\n            score: item.score != null ? Number(item.score) : null,\n            author: String(item.submitter_user ?? ''),\n            comments: item.comment_count != null ? Number(item.comment_count) : null,\n            created_at: String(item.created_at ?? '').slice(0, 10),\n            tags: Array.isArray(item.tags) ? item.tags.join(', ') : '',\n            submission_url: String(item.url ?? ''),\n            comments_url: String(item.comments_url ?? ''),\n        }));\n    },","sourceCodeStart":55,"sourceCodeEnd":91,"githubUrl":"https://github.com/jackwener/OpenCLI/blob/49907e53dc3ade5c223ff0c4c2c2785687cec4e6/clis/lobsters/domain.js#L55-L91","documentation":"This CommandExecutionError is thrown when the HTTP response body from the lobste.rs domain endpoint cannot be parsed as JSON (resp.json() rejects). Since a non-ok status was already handled, this means the server returned a 2xx/3xx response whose body is not valid JSON — typically an HTML error/challenge page. The original parse error message is appended to help diagnose the malformed payload.","triggerScenarios":"lobste.rs (or an intermediary proxy/CDN) returns HTTP 200 with an HTML body: Cloudflare bot-challenge or 'checking your browser' page, a captive-portal login page, a maintenance page served with 200, or a truncated/corrupted response body.","commonSituations":"Running from datacenter IPs or CI runners that Cloudflare challenges, corporate proxies that rewrite responses, offline captive networks that intercept all HTTP with a login page, or a lobste.rs incident where the API serves HTML.","solutions":["Fetch the endpoint manually (curl -i https://lobste.rs/domain/<domain>.json) and inspect whether the body is HTML from a challenge page or proxy.","Set a descriptive User-Agent header if you control the request, as Cloudflare frequently challenges requests with bot-like or missing User-Agents.","Retry from a different network (residential IP instead of datacenter/CI) to rule out Cloudflare/proxy interception.","Catch this error and surface the inner message plus a snippet of the raw body to confirm what was actually returned."],"exampleFix":"// before\n// resp.json() fails on a Cloudflare HTML page; you only see 'malformed JSON'\n\n// after\nconst text = await resp.text();\nlet body;\ntry {\n  body = JSON.parse(text);\n} catch (err) {\n  console.error('lobsters domain returned non-JSON body:', text.slice(0, 200));\n  throw err;\n}","handlingStrategy":"try-catch","validationCode":"async function looksLikeJsonEndpoint(url) {\n  try {\n    const r = await fetch(url, { headers: { Accept: 'application/json' } });\n    const ct = r.headers.get('content-type') || '';\n    return ct.includes('application/json');\n  } catch {\n    return false;\n  }\n}","typeGuard":"null","tryCatchPattern":"try {\n  const stories = await cli.lobsters.domain(domain);\n} catch (err) {\n  if (/malformed JSON/.test(String(err.message))) {\n    console.error('lobste.rs returned a non-JSON body (likely Cloudflare challenge or proxy page). Try a different network or check https://lobste.rs directly.');\n    return;\n  }\n  throw err;\n}","preventionTips":["Check the response content-type is application/json before parsing in your own fetches.","Avoid calling from datacenter/CI IPs that commonly receive Cloudflare challenges; use a normal User-Agent.","Detect captive portals: if other HTTPS sites return HTML challenges, pause API calls.","Log a snippet of the raw body when JSON parsing fails to identify who injected the HTML."],"tags":["network","json-parsing","cloudflare","cli"],"backgroundTag":"invalid-json-response","analyzedSha":"49907e53dc3ade5c223ff0c4c2c2785687cec4e6","analyzedAt":"2026-08-29T08:14:47.543Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}