{"record":{"id":"50b7991877b4c434","repo":"jackwener/OpenCLI","slug":"xiaohongshu-collection-api-returned-a-malformed-pa","errorCode":null,"errorMessage":"xiaohongshu collection API returned a malformed payload","messagePattern":"xiaohongshu collection API returned a malformed payload","errorType":"exception","errorClass":"CommandExecutionError","httpStatus":null,"severity":"error","filePath":"clis/xiaohongshu/collection-helpers.js","lineNumber":91,"sourceCode":"        ? buildXhsNoteUrl(userId, noteId, xsecToken)\n        : `https://www.xiaohongshu.com/explore/${encodeURIComponent(noteId)}?xsec_token=${encodeURIComponent(xsecToken)}&xsec_source=pc_user`;\n    return {\n        id: noteId,\n        title: toCleanString(noteCard.display_title ?? noteCard.displayTitle ?? noteCard.title ?? entry.title ?? entry.display_title),\n        author: toCleanString(user.nickname ?? user.nick_name ?? user.name),\n        likes: toCleanString(interact.liked_count ?? interact.likedCount ?? 0) || '0',\n        type: toCleanString(noteCard.type ?? entry.type),\n        url,\n    };\n}\n\nexport function extractNotesFromResponses(requests, fallbackUserId) {\n    const rows = [];\n    const seen = new Set();\n    for (const req of requests ?? []) {\n        const payload = unwrapBrowserResult(req);\n        if (!isObject(payload)) {\n            throw new CommandExecutionError('xiaohongshu collection API returned a malformed payload');\n        }\n        const data = payload.data;\n        if (!isObject(data)) {\n            throw new CommandExecutionError('xiaohongshu collection API returned malformed data');\n        }\n        const notes = data.notes ?? data.note_list;\n        if (!Array.isArray(notes))\n            throw new CommandExecutionError('xiaohongshu collection API returned malformed notes');\n        for (const entry of notes) {\n            const row = mapCollectionNote(entry, { fallbackUserId });\n            if (!row?.id || !row.url.includes('xsec_token=')) {\n                throw new CommandExecutionError('xiaohongshu collection API returned a note without stable id/xsec token');\n            }\n            if (seen.has(row.id))\n                continue;\n            seen.add(row.id);\n            rows.push(row);\n        }","sourceCodeStart":73,"sourceCodeEnd":109,"githubUrl":"https://github.com/jackwener/OpenCLI/blob/49907e53dc3ade5c223ff0c4c2c2785687cec4e6/clis/xiaohongshu/collection-helpers.js#L73-L109","documentation":"extractNotesFromResponses unwraps each intercepted collection API response via unwrapBrowserResult and requires the result to be a plain object. If a captured response does not unwrap to an object, the helper throws CommandExecutionError because it cannot interpret the payload shape. This guards against interceptor captures of non-JSON or error responses.","triggerScenarios":"A network request matching the collection API pattern returns HTML (login page, CAPTCHA page, error page), an empty body, or a string/array instead of a JSON object; unwrapBrowserResult then yields a non-object and this error fires.","commonSituations":"Xiaohongshu serving an anti-bot or WAF HTML interstitial instead of JSON; expired session returning a redirect page; CDN error page captured by the interceptor.","solutions":["Re-login to xiaohongshu.com in the automation browser and retry","Inspect page.getInterceptedRequests() and check response bodies for HTML/anti-bot pages","Add retry with delay before scraping to avoid rate-limit interstitials","Verify your network interception URL filter matches only the real collection API endpoint"],"exampleFix":"// before\nconst rows = extractNotesFromResponses(await page.getInterceptedRequests(), userId); // throws on HTML capture\n// after\nconst reqs = (await page.getInterceptedRequests()).filter(r => {\n  try { return typeof r.json === 'object' && r.json !== null; } catch { return false; }\n});\nconst rows = extractNotesFromResponses(reqs, userId);","handlingStrategy":"type-guard","validationCode":"const isPlainObject = (v) => v !== null && typeof v === 'object' && !Array.isArray(v);","typeGuard":"function isObjectPayload(v) { return v !== null && typeof v === 'object' && !Array.isArray(v); }","tryCatchPattern":"try { rows = extractNotesFromResponses(reqs, userId); } catch (e) { if (String(e.message).includes('malformed payload')) { await relogin(page); rows = retry(); } else throw e; }","preventionTips":["Filter intercepted captures to JSON content-type responses before extraction","Log raw response bodies when extraction fails to spot HTML interstitials","Detect anti-bot/login HTML early and refresh the session","Keep the interceptor URL filter narrow so only API captures reach the extractor"],"tags":["scraping","malformed-response","network-interception","command-execution-error"],"backgroundTag":"malformed-api-response","analyzedSha":"49907e53dc3ade5c223ff0c4c2c2785687cec4e6","analyzedAt":"2026-08-29T08:14:47.543Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}