{"record":{"id":"73a30e86ae6d906e","repo":"jackwener/OpenCLI","slug":"xiaohongshu-media-extraction-returned-malformed-pa","errorCode":null,"errorMessage":"Xiaohongshu media extraction returned malformed payload.","messagePattern":"Xiaohongshu media extraction returned malformed payload\\.","errorType":"exception","errorClass":"CommandExecutionError","httpStatus":null,"severity":"error","filePath":"clis/xiaohongshu/download.js","lineNumber":231,"sourceCode":"    args: [\n        { name: 'note-id', positional: true, required: true, help: 'Full Xiaohongshu note URL with xsec_token, or xhslink short link' },\n        { name: 'output', default: './xiaohongshu-downloads', help: 'Output directory' },\n    ],\n    columns: ['index', 'type', 'status', 'size'],\n    func: async (page, kwargs) => {\n        const rawInput = String(kwargs['note-id']);\n        const output = kwargs.output;\n        const noteId = parseNoteId(rawInput);\n        await page.goto(buildNoteUrl(rawInput, { allowShortLink: true, commandName: 'xiaohongshu download' }));\n        await page.wait({ time: 1 + Math.random() * 2 });\n        const data = await page.evaluate(buildDownloadExtractJs(noteId));\n        if (data?.securityBlock) {\n            throw new CliError('SECURITY_BLOCK', 'Xiaohongshu security block: the note detail page was blocked by risk control.', /^https?:\\/\\//.test(rawInput)\n                ? 'The page may be temporarily restricted. Try again later or from a different session.'\n                : 'Try using a full URL from search results (with xsec_token) instead of a bare note ID.');\n        }\n        if (!data || typeof data !== 'object' || !Array.isArray(data.media)) {\n            throw new CommandExecutionError('Xiaohongshu media extraction returned malformed payload.');\n        }\n        if (data.media.length === 0) {\n            throw new EmptyResultError('xiaohongshu download', 'No downloadable media found on this note.');\n        }\n        // Extract cookies for authenticated downloads\n        const cookies = formatCookieHeader(await page.getCookies({ domain: 'xiaohongshu.com' }));\n        const resolvedNoteId = typeof data.noteId === 'string' && data.noteId.trim()\n            ? data.noteId.trim()\n            : noteId;\n        return downloadMedia(data.media, {\n            output,\n            subdir: resolvedNoteId,\n            cookies,\n            filenamePrefix: resolvedNoteId,\n            timeout: 60000,\n        });\n    },\n});","sourceCodeStart":213,"sourceCodeEnd":249,"githubUrl":"https://github.com/jackwener/OpenCLI/blob/49907e53dc3ade5c223ff0c4c2c2785687cec4e6/clis/xiaohongshu/download.js#L213-L249","documentation":"CommandExecutionError thrown when the page.evaluate extraction (buildDownloadExtractJs) returns something that is not an object with a `media` array. This means the extraction script did not run in the expected page context or the page DOM changed so the script returned null/undefined/a primitive. It guards downstream code that assumes `data.media` is iterable.","triggerScenarios":"page.evaluate returns null or a non-object because the note page did not render the expected structure; the extraction script threw silently and returned undefined; a DOM redesign changed the fields buildDownloadExtractJs reads; the script ran on a login/redirect page instead of the note page.","commonSituations":"Xiaohongshu shipped a front-end update that broke the extractor; the page redirected to login or a captcha (without setting securityBlock); a race where evaluate ran before the note content mounted.","solutions":["Re-run once — it may be a transient render race; increase the pre-evaluate wait.","Confirm you can see the note normally in a browser session with the same cookies.","Inspect the raw evaluate return to see what the page actually returned and update buildDownloadExtractJs selectors for the new DOM.","Ensure the input URL actually resolves to a note detail page (not a profile or redirect)."],"exampleFix":"// before\nawait page.goto(url);\nconst data = await page.evaluate(buildDownloadExtractJs(noteId));\n// after\nawait page.goto(url);\nawait page.wait({ time: 2 + Math.random() * 2 }); // let note content mount\nconst data = await page.evaluate(buildDownloadExtractJs(noteId));\nif (!data || !Array.isArray(data?.media)) throw new Error('unexpected page: ' + await page.title());","handlingStrategy":"type-guard","validationCode":"null","typeGuard":"function isExtractPayload(data) {\n  return data !== null && typeof data === 'object' && Array.isArray(data.media);\n}","tryCatchPattern":"try {\n  await cli.run('xiaohongshu download', { input: url });\n} catch (err) {\n  if (/malformed payload/.test(err.message)) {\n    // page didn't render as expected: increase wait, retry once, then surface\n    await sleep(3000);\n    return downloadNote(url);\n  }\n  throw err;\n}","preventionTips":["Keep the extraction script's selectors in sync with the current Xiaohongshu DOM.","Add a settle wait before page.evaluate so dynamic note content has mounted.","Check for login/captcha redirects before running the extractor.","Validate the payload shape immediately after evaluate to fail fast with context."],"tags":["extraction","dom-parse","malformed-payload","web-automation"],"backgroundTag":"malformed-extraction-payload","analyzedSha":"49907e53dc3ade5c223ff0c4c2c2785687cec4e6","analyzedAt":"2026-08-29T08:14:47.543Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}