jackwener/OpenCLI · error · CommandExecutionError

Zhihu ${label} pagination returned a repeated next URL

Error message

Zhihu ${label} pagination returned a repeated next URL

What it means

fetchPages tracks every paging URL it has requested. If paging.next leads back to an already-visited URL, the API's pagination is looping and would never terminate, so the library throws. This is a guard against Zhihu's paging.next returning a stale or self-referential link.

Source

Thrown at clis/zhihu/answer-comments-helpers.js:139

        if (previous.descriptor.signature !== descriptor.signature) {
            throw new CommandExecutionError(`Zhihu answer comments returned conflicting data for comment ${descriptor.id}`);
        }
        const oldCount = previous.comment.child_comment_count;
        const newCount = comment.child_comment_count;
        if (Number.isInteger(newCount) && (!Number.isInteger(oldCount) || newCount > oldCount)) {
            previous.comment = { ...previous.comment, child_comment_count: newCount };
        }
    }
}
async function fetchPages(page, options) {
    const { firstUrl, limit, label, role, expectedRootId = '', normalizeNext, notFoundDetail = '' } = options;
    const byId = new Map();
    const visited = new Set();
    const maxPages = Math.ceil(limit / PAGE_SIZE) + PAGE_OVERLAP_ALLOWANCE;
    let pageCount = 0;
    let url = firstUrl;
    while (byId.size < limit) {
        if (visited.has(url)) throw new CommandExecutionError(`Zhihu ${label} pagination returned a repeated next URL`);
        if (pageCount >= maxPages) throw new CommandExecutionError(`Zhihu ${label} pagination exceeded its fetch budget`);
        visited.add(url);
        pageCount += 1;
        const payload = await fetchCommentPage(page, url, label, notFoundDetail);
        addPageComments(byId, payload.data, role, expectedRootId);
        if (payload.paging.is_end || byId.size >= limit) break;
        url = normalizeNext(payload.paging.next);
        if (!url) throw new CommandExecutionError(`Zhihu ${label} pagination returned a malformed next URL`);
    }
    return [...byId.values()].slice(0, limit).map(({ comment }) => comment);
}
export function fetchRootComments(page, answerId, order, limit) {
    const apiOrder = order === 'latest' ? 'ts' : 'score';
    const path = `/api/v4/comment_v5/answers/${answerId}/root_comment`;
    return fetchPages(page, {
        firstUrl: `https://www.zhihu.com${path}?order_by=${apiOrder}&limit=${PAGE_SIZE}&offset=`,
        limit,
        label: 'answer root comments',

View on GitHub (pinned to 49907e53dc)

Solutions

  1. Print each requested URL to identify which cursor link loops back and what param is missing.
  2. Verify the order argument ('latest' vs score) is supported; try the other order.
  3. Bypass any proxy/rewriter that strips or reorders query parameters (the cursor token is part of the URL).
  4. Retry later if it's a transient Zhihu pagination bug; check whether the answer's comment count changed mid-fetch.

Example fix

// before: rewriting next through a proxy that drops the cursor
const next = resp.paging.next.replace('https://api.zhihu.com', MY_PROXY); // cursor lost
// after: pass the cursor query through untouched
const next = normalizeNext(resp.paging.next);
Defensive patterns

Strategy: retry

Validate before calling

// sanity-check a pagination stream yourself before trusting it
function detectLoop(urls) {
  const seen = new Set();
  for (const u of urls) {
    if (seen.has(u)) return u;
    seen.add(u);
  }
  return null;
}

Try / catch

try {
  const comments = await fetchRootComments(page, answerId, order, limit);
} catch (err) {
  if (String(err.message).includes('repeated next URL')) {
    console.warn('Zhihu pagination looped; retrying with the other sort order');
    return fetchRootComments(page, answerId, order === 'latest' ? 'score' : 'latest', limit);
  }
  throw err;
}

Prevention

When it happens

Trigger: payload.paging.next normalizes (normalizeNext) to a URL already in the visited set — typically because is_end never became true and next kept pointing at the same or an earlier page.

Common situations: Zhihu API bug for a given sort order (e.g. order param not honored so the cursor never advances), proxy stripping cursor query params so next resolves to page 1, or an incorrect answerId/order combination making the server return a static next link.

Related errors

AI-assisted analysis of jackwener/OpenCLI@49907e53dc (2026-08-29). Data as JSON: /api/errors/2444129f107deb26. Report an issue: GitHub.