{"record":{"id":"449d937017b050f0","repo":"jackwener/OpenCLI","slug":"zhihu-label-pagination-returned-a-malformed-nex","errorCode":null,"errorMessage":"Zhihu ${label} pagination returned a malformed next URL","messagePattern":"Zhihu (.+?) pagination returned a malformed next URL","errorType":"exception","errorClass":"CommandExecutionError","httpStatus":null,"severity":"error","filePath":"clis/zhihu/answer-comments-helpers.js","lineNumber":147,"sourceCode":"    }\n}\nasync function fetchPages(page, options) {\n    const { firstUrl, limit, label, role, expectedRootId = '', normalizeNext, notFoundDetail = '' } = options;\n    const byId = new Map();\n    const visited = new Set();\n    const maxPages = Math.ceil(limit / PAGE_SIZE) + PAGE_OVERLAP_ALLOWANCE;\n    let pageCount = 0;\n    let url = firstUrl;\n    while (byId.size < limit) {\n        if (visited.has(url)) throw new CommandExecutionError(`Zhihu ${label} pagination returned a repeated next URL`);\n        if (pageCount >= maxPages) throw new CommandExecutionError(`Zhihu ${label} pagination exceeded its fetch budget`);\n        visited.add(url);\n        pageCount += 1;\n        const payload = await fetchCommentPage(page, url, label, notFoundDetail);\n        addPageComments(byId, payload.data, role, expectedRootId);\n        if (payload.paging.is_end || byId.size >= limit) break;\n        url = normalizeNext(payload.paging.next);\n        if (!url) throw new CommandExecutionError(`Zhihu ${label} pagination returned a malformed next URL`);\n    }\n    return [...byId.values()].slice(0, limit).map(({ comment }) => comment);\n}\nexport function fetchRootComments(page, answerId, order, limit) {\n    const apiOrder = order === 'latest' ? 'ts' : 'score';\n    const path = `/api/v4/comment_v5/answers/${answerId}/root_comment`;\n    return fetchPages(page, {\n        firstUrl: `https://www.zhihu.com${path}?order_by=${apiOrder}&limit=${PAGE_SIZE}&offset=`,\n        limit,\n        label: 'answer root comments',\n        role: 'root',\n        normalizeNext: (next) => normalizePageUrl(next, path, {\n            order_by: apiOrder,\n            limit: String(PAGE_SIZE),\n            offset: null,\n        }),\n        notFoundDetail: `No Zhihu answer comments resource was found for ${answerId}.`,\n    });","sourceCodeStart":129,"sourceCodeEnd":165,"githubUrl":"https://github.com/jackwener/OpenCLI/blob/49907e53dc3ade5c223ff0c4c2c2785687cec4e6/clis/zhihu/answer-comments-helpers.js#L129-L165","documentation":"After a page that is not is_end, fetchPages normalizes payload.paging.next with normalizeNext; if the result is falsy the next link is absent or malformed, making continuation impossible. The library throws instead of silently truncating results.","triggerScenarios":"A non-final page whose paging.next is null, an empty string, a relative URL normalizeNext cannot resolve, or a URL failing normalization (bad scheme/host).","commonSituations":"Zhihu changing the next-link format (e.g. protocol-relative or a new host), proxies rewriting the URL into something normalizeNext rejects, or API responses truncated mid-field by a flaky network layer.","solutions":["Log payload.paging.next on the failing page and check what normalizeNext rejects about it.","Update/patch the library's normalizeNext to accept the new URL shape if Zhihu changed formats.","Ensure proxies pass the next URL through verbatim; disable URL rewriting middleware.","Retry the fetch in case the page was truncated by a transient network error."],"exampleFix":"// before: proxy rewrites https to http and normalizeNext rejects\nconst page = await fetch(nextUrl.replace('https://', 'http://'));\n// after: keep the original URL\nconst page = await fetch(nextUrl);","handlingStrategy":"try-catch","validationCode":"// pre-validate paging payloads before handing them to the walker\nfunction nextUrlIsUsable(payload) {\n  if (payload?.paging?.is_end) return true;\n  const next = payload?.paging?.next;\n  return typeof next === 'string' && /^https?:\\/\\//.test(next);\n}","typeGuard":"function isAbsoluteHttpUrl(v) {\n  if (typeof v !== 'string' || v === '') return false;\n  try { const u = new URL(v); return u.protocol === 'https:' || u.protocol === 'http:'; }\n  catch { return false; }\n}","tryCatchPattern":"try {\n  const comments = await fetchRootComments(page, answerId, order, limit);\n} catch (err) {\n  if (String(err.message).includes('malformed next URL')) {\n    console.warn('Paging stopped at a bad next link; keeping partial results');\n    return partialResultsSoFar;\n  }\n  throw err;\n}","preventionTips":["Inspect paging.next on the first page and fail fast if its shape is unexpected.","Disable URL-rewriting middleware/proxies for Zhihu endpoints.","Keep the library updated for normalizeNext changes when Zhihu alters URL formats."],"tags":["pagination","api","url","zhihu"],"backgroundTag":"malformed-next-url","analyzedSha":"49907e53dc3ade5c223ff0c4c2c2785687cec4e6","analyzedAt":"2026-08-29T08:14:47.543Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}