jackwener/OpenCLI · error · CommandExecutionError

xiaohongshu collection API returned a malformed payload

Error message

xiaohongshu collection API returned a malformed payload

What it means

extractNotesFromResponses unwraps each intercepted collection API response via unwrapBrowserResult and requires the result to be a plain object. If a captured response does not unwrap to an object, the helper throws CommandExecutionError because it cannot interpret the payload shape. This guards against interceptor captures of non-JSON or error responses.

Source

Thrown at clis/xiaohongshu/collection-helpers.js:91

        ? buildXhsNoteUrl(userId, noteId, xsecToken)
        : `https://www.xiaohongshu.com/explore/${encodeURIComponent(noteId)}?xsec_token=${encodeURIComponent(xsecToken)}&xsec_source=pc_user`;
    return {
        id: noteId,
        title: toCleanString(noteCard.display_title ?? noteCard.displayTitle ?? noteCard.title ?? entry.title ?? entry.display_title),
        author: toCleanString(user.nickname ?? user.nick_name ?? user.name),
        likes: toCleanString(interact.liked_count ?? interact.likedCount ?? 0) || '0',
        type: toCleanString(noteCard.type ?? entry.type),
        url,
    };
}

export function extractNotesFromResponses(requests, fallbackUserId) {
    const rows = [];
    const seen = new Set();
    for (const req of requests ?? []) {
        const payload = unwrapBrowserResult(req);
        if (!isObject(payload)) {
            throw new CommandExecutionError('xiaohongshu collection API returned a malformed payload');
        }
        const data = payload.data;
        if (!isObject(data)) {
            throw new CommandExecutionError('xiaohongshu collection API returned malformed data');
        }
        const notes = data.notes ?? data.note_list;
        if (!Array.isArray(notes))
            throw new CommandExecutionError('xiaohongshu collection API returned malformed notes');
        for (const entry of notes) {
            const row = mapCollectionNote(entry, { fallbackUserId });
            if (!row?.id || !row.url.includes('xsec_token=')) {
                throw new CommandExecutionError('xiaohongshu collection API returned a note without stable id/xsec token');
            }
            if (seen.has(row.id))
                continue;
            seen.add(row.id);
            rows.push(row);
        }

View on GitHub (pinned to 49907e53dc)

Solutions

1. Re-login to xiaohongshu.com in the automation browser and retry
2. Inspect page.getInterceptedRequests() and check response bodies for HTML/anti-bot pages
3. Add retry with delay before scraping to avoid rate-limit interstitials
4. Verify your network interception URL filter matches only the real collection API endpoint

Example fix

// before
const rows = extractNotesFromResponses(await page.getInterceptedRequests(), userId); // throws on HTML capture
// after
const reqs = (await page.getInterceptedRequests()).filter(r => {
  try { return typeof r.json === 'object' && r.json !== null; } catch { return false; }
});
const rows = extractNotesFromResponses(reqs, userId);

const isPlainObject = (v) => v !== null && typeof v === 'object' && !Array.isArray(v);

function isObjectPayload(v) { return v !== null && typeof v === 'object' && !Array.isArray(v); }

try { rows = extractNotesFromResponses(reqs, userId); } catch (e) { if (String(e.message).includes('malformed payload')) { await relogin(page); rows = retry(); } else throw e; }

When it happens

Trigger: A network request matching the collection API pattern returns HTML (login page, CAPTCHA page, error page), an empty body, or a string/array instead of a JSON object; unwrapBrowserResult then yields a non-object and this error fires.

Common situations: Xiaohongshu serving an anti-bot or WAF HTML interstitial instead of JSON; expired session returning a redirect page; CDN error page captured by the interceptor.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• xiaohongshu collection API returned malformed data
• xiaohongshu collection API returned malformed notes
• xiaohongshu collection interceptor returned malformed captur
• ${label} returned malformed JSON: ${err?.message ?? err}
• Pixiv novel ${id} returned malformed detail payload

AI-assisted analysis of jackwener/OpenCLI@49907e53dc (2026-08-29). Data as JSON: /api/errors/50b7991877b4c434. Report an issue: GitHub.