jackwener/OpenCLI · error · CommandExecutionError

toutiao hot-board returned malformed JSON: ${error?.message

Error message

toutiao hot-board returned malformed JSON: ${error?.message || error}

What it means

The command calls resp.json() on the hot-board response; if the body cannot be parsed as JSON this wraps the parse error in a CommandExecutionError. It means the server returned 200 but the body is HTML (e.g. a captcha/verification page), empty, or truncated instead of the expected JSON.

Source

Thrown at clis/toutiao/hot.js:47

        try {
            resp = await fetch(HOT_BOARD_URL, {
                headers: {
                    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36',
                    Accept: 'application/json',
                    Referer: 'https://www.toutiao.com/',
                },
            });
        } catch (error) {
            throw new CommandExecutionError(`toutiao hot-board request failed: ${error?.message || error}`);
        }
        if (!resp.ok) {
            throw new CommandExecutionError(`toutiao hot-board failed: HTTP ${resp.status}`);
        }
        let payload;
        try {
            payload = await resp.json();
        } catch (error) {
            throw new CommandExecutionError(`toutiao hot-board returned malformed JSON: ${error?.message || error}`);
        }
        if (payload?.status && payload.status !== 'success') {
            throw new CommandExecutionError(`toutiao hot-board returned status=${payload.status}`);
        }
        if (payload?.error || payload?.message) {
            throw new CommandExecutionError(`toutiao hot-board returned error: ${payload.error || payload.message}`);
        }
        const list = Array.isArray(payload?.data) ? payload.data : [];
        const rows = list.map(mapHotRow).filter(Boolean).slice(0, limit);
        if (rows.length === 0) {
            throw new EmptyResultError('toutiao hot', '上游 hot-board 返回空列表。');
        }
        // Re-rank (1..N) after filter so ranks are dense even if upstream had nulls.
        return rows.map((row, idx) => ({ ...row, rank: idx + 1 }));
    },
});

View on GitHub (pinned to 49907e53dc)

Solutions

Log resp.headers.get('content-type') and the first ~200 chars of resp.text() on failure to see what the server actually returned.
If it's an anti-bot HTML page, send the full browser-like headers (User-Agent, Referer, Accept) and consider a headless-browser strategy.
Retry the request — truncated/corrupted bodies are often transient.
Check whether a proxy in the path is rewriting the response; bypass it.

Example fix

// before
try {
  payload = await resp.json();
} catch (error) {
  throw new CommandExecutionError(`toutiao hot-board returned malformed JSON: ${error?.message || error}`);
}
// after — include a body preview for diagnosis
let raw = await resp.text();
let payload;
try {
  payload = JSON.parse(raw);
} catch (error) {
  throw new CommandExecutionError(`toutiao hot-board returned malformed JSON: ${error?.message || error}; body: ${raw.slice(0, 200)}`);
}

Defensive patterns

Strategy: validation

Validate before calling

// validate content-type before parsing
const ct = resp.headers.get('content-type') || '';
if (!ct.includes('application/json')) {
  throw new Error(`expected JSON, got ${ct}; endpoint likely serving an anti-bot page`);
}

Try / catch

try {
  const rows = await toutiaoHot({ limit: 30 });
} catch (e) {
  if (/malformed JSON/.test(e.message)) {
    // server sent HTML/empty body — log headers, back off, retry once
    await sleep(1500);
    return toutiaoHot({ limit: 30 });
  }
  throw e;
}

Prevention

Check the content-type header before calling resp.json().
Capture a body preview on parse failure to detect anti-bot HTML pages.
Keep browser-like headers current so the WAF serves JSON, not a verification page.
Detect and log non-JSON responses in monitoring to catch endpoint/CDN changes early.

When it happens

Trigger: Calling `toutiao hot` when the endpoint returns an anti-bot HTML interstitial, an error page, an empty body, or a gzip/encoding-corrupted response — any body that JSON.parse fails on.

Common situations: Toutiao's WAF serves an HTML 'verify' or login page to suspicious clients; a proxy/CDN injects an HTML error page; the response is truncated by a flaky network layer; Accept-Encoding handling in a custom proxy mangles the body.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of jackwener/OpenCLI@49907e53dc (2026-08-29). Data as JSON: /api/errors/6c0ee5b0e72a5944. Report an issue: GitHub.