{"record":{"id":"6c0ee5b0e72a5944","repo":"jackwener/OpenCLI","slug":"toutiao-hot-board-returned-malformed-json-error","errorCode":null,"errorMessage":"toutiao hot-board returned malformed JSON: ${error?.message || error}","messagePattern":"toutiao hot-board returned malformed JSON: (.+?)","errorType":"exception","errorClass":"CommandExecutionError","httpStatus":null,"severity":"error","filePath":"clis/toutiao/hot.js","lineNumber":47,"sourceCode":"        try {\n            resp = await fetch(HOT_BOARD_URL, {\n                headers: {\n                    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36',\n                    Accept: 'application/json',\n                    Referer: 'https://www.toutiao.com/',\n                },\n            });\n        } catch (error) {\n            throw new CommandExecutionError(`toutiao hot-board request failed: ${error?.message || error}`);\n        }\n        if (!resp.ok) {\n            throw new CommandExecutionError(`toutiao hot-board failed: HTTP ${resp.status}`);\n        }\n        let payload;\n        try {\n            payload = await resp.json();\n        } catch (error) {\n            throw new CommandExecutionError(`toutiao hot-board returned malformed JSON: ${error?.message || error}`);\n        }\n        if (payload?.status && payload.status !== 'success') {\n            throw new CommandExecutionError(`toutiao hot-board returned status=${payload.status}`);\n        }\n        if (payload?.error || payload?.message) {\n            throw new CommandExecutionError(`toutiao hot-board returned error: ${payload.error || payload.message}`);\n        }\n        const list = Array.isArray(payload?.data) ? payload.data : [];\n        const rows = list.map(mapHotRow).filter(Boolean).slice(0, limit);\n        if (rows.length === 0) {\n            throw new EmptyResultError('toutiao hot', '上游 hot-board 返回空列表。');\n        }\n        // Re-rank (1..N) after filter so ranks are dense even if upstream had nulls.\n        return rows.map((row, idx) => ({ ...row, rank: idx + 1 }));\n    },\n});\n","sourceCodeStart":29,"sourceCodeEnd":64,"githubUrl":"https://github.com/jackwener/OpenCLI/blob/49907e53dc3ade5c223ff0c4c2c2785687cec4e6/clis/toutiao/hot.js#L29-L64","documentation":"The command calls resp.json() on the hot-board response; if the body cannot be parsed as JSON this wraps the parse error in a CommandExecutionError. It means the server returned 200 but the body is HTML (e.g. a captcha/verification page), empty, or truncated instead of the expected JSON.","triggerScenarios":"Calling `toutiao hot` when the endpoint returns an anti-bot HTML interstitial, an error page, an empty body, or a gzip/encoding-corrupted response — any body that JSON.parse fails on.","commonSituations":"Toutiao's WAF serves an HTML 'verify' or login page to suspicious clients; a proxy/CDN injects an HTML error page; the response is truncated by a flaky network layer; Accept-Encoding handling in a custom proxy mangles the body.","solutions":["Log resp.headers.get('content-type') and the first ~200 chars of resp.text() on failure to see what the server actually returned.","If it's an anti-bot HTML page, send the full browser-like headers (User-Agent, Referer, Accept) and consider a headless-browser strategy.","Retry the request — truncated/corrupted bodies are often transient.","Check whether a proxy in the path is rewriting the response; bypass it."],"exampleFix":"// before\ntry {\n  payload = await resp.json();\n} catch (error) {\n  throw new CommandExecutionError(`toutiao hot-board returned malformed JSON: ${error?.message || error}`);\n}\n// after — include a body preview for diagnosis\nlet raw = await resp.text();\nlet payload;\ntry {\n  payload = JSON.parse(raw);\n} catch (error) {\n  throw new CommandExecutionError(`toutiao hot-board returned malformed JSON: ${error?.message || error}; body: ${raw.slice(0, 200)}`);\n}","handlingStrategy":"validation","validationCode":"// validate content-type before parsing\nconst ct = resp.headers.get('content-type') || '';\nif (!ct.includes('application/json')) {\n  throw new Error(`expected JSON, got ${ct}; endpoint likely serving an anti-bot page`);\n}","typeGuard":null,"tryCatchPattern":"try {\n  const rows = await toutiaoHot({ limit: 30 });\n} catch (e) {\n  if (/malformed JSON/.test(e.message)) {\n    // server sent HTML/empty body — log headers, back off, retry once\n    await sleep(1500);\n    return toutiaoHot({ limit: 30 });\n  }\n  throw e;\n}","preventionTips":["Check the content-type header before calling resp.json().","Capture a body preview on parse failure to detect anti-bot HTML pages.","Keep browser-like headers current so the WAF serves JSON, not a verification page.","Detect and log non-JSON responses in monitoring to catch endpoint/CDN changes early."],"tags":["json","parse-error","anti-bot","http"],"backgroundTag":"malformed-json-response","analyzedSha":"49907e53dc3ade5c223ff0c4c2c2785687cec4e6","analyzedAt":"2026-08-29T08:14:47.543Z","schemaVersion":2},"datasetVersion":"2026-08-29T17:17:51.833Z"}