{"record":{"id":"b3803e0da8035868","repo":"jackwener/OpenCLI","slug":"twitter-collection-page-guard-hit","errorCode":"twitter_collection_page_guard_hit","errorMessage":"twitter_collection_page_guard_hit: pagination cannot prove completion","messagePattern":"twitter_collection_page_guard_hit: pagination cannot prove completion","errorType":"error_code","errorClass":"CommandExecutionError","httpStatus":null,"severity":"error","filePath":"clis/twitter/collection.js","lineNumber":297,"sourceCode":"                };\n            }\n        }\n        if (!nextCursor) {\n            return {\n                posts,\n                receipt: completedReceipt('cursor_exhausted', until, pageIndex + 1, oldestSeenAt),\n            };\n        }\n        if (nextCursor === cursor || seenCursors.has(nextCursor)) {\n            throw new CommandExecutionError('twitter_collection_repeated_cursor: pagination cannot prove completion');\n        }\n        if (posts.length >= limit) {\n            throw new CommandExecutionError('twitter_collection_limit_reached: pagination cannot prove completion');\n        }\n        seenCursors.add(nextCursor);\n        cursor = nextCursor;\n    }\n    throw new CommandExecutionError('twitter_collection_page_guard_hit: pagination cannot prove completion');\n}\n\ncli({\n    site: 'twitter',\n    name: 'collection',\n    access: 'read',\n    description: 'Fetch a user timeline with relationship facts and a bounded completion receipt.',\n    domain: 'x.com',\n    strategy: Strategy.COOKIE,\n    browser: true,\n    args: [\n        { name: 'username', type: 'string', positional: true, required: true, help: 'Twitter screen name (with or without @).' },\n        { name: 'until', type: 'string', required: true, help: 'RFC3339 lower time boundary that must be reached or exhausted.' },\n        { name: 'limit', type: 'int', default: MAX_USER_TWEETS_LIMIT, help: 'Safety ceiling; reaching it is a typed failure.' },\n        { name: 'page-delay', type: 'int', default: DEFAULT_USER_TWEETS_PAGE_DELAY_SECONDS, help: 'Seconds to wait between cursor pages.' },\n    ],\n    columns: ['posts', 'receipt'],\n    func: async (page, kwargs) => {","sourceCodeStart":279,"sourceCodeEnd":315,"githubUrl":"https://github.com/jackwener/OpenCLI/blob/49907e53dc3ade5c223ff0c4c2c2785687cec4e6/clis/twitter/collection.js#L279-L315","documentation":"If the for-loop over pageIndex completes maxPages iterations without reaching 'until' or exhausting the cursor, the pagination guard trips: the page budget ran out before completion could be proven. The library throws rather than returning an unverified, potentially gap-filled post set.","triggerScenarios":"maxPages is smaller than the number of pages needed to walk from the newest post back to 'until' — common for high-volume accounts, old cutoffs, or small page sizes (USER_TWEETS_PAGE_SIZE).","commonSituations":"First-time deep backfill of a long-running account; 'until' set months back; rate-limit backoff shrinking usable pages; accounts with gaps/deleted tweets inflating page counts.","solutions":["Increase maxPages so the page budget covers posts between now and 'until'.","Narrow 'until' to a recent timestamp and collect incrementally across runs.","Raise USER_TWEETS_PAGE_SIZE if X still honors it, reducing the number of pages required.","Perform an initial full backfill once, then run frequent small incremental collections.","Catch the error and persist oldestSeenAt from the receipt-building path to resume later."],"exampleFix":"// before\nconst { posts } = await collection({ user: 'handle', until: '2026-01-01', maxPages: 5 });\n// after\nconst { posts } = await collection({ user: 'handle', until: '2026-01-01', maxPages: 200 });","handlingStrategy":"try-catch","validationCode":"const expectedPages = Math.ceil(expectedPosts / USER_TWEETS_PAGE_SIZE);\nif (expectedPages > maxPages) {\n  console.warn(`maxPages ${maxPages} < estimated ${expectedPages} pages needed`);\n}","typeGuard":null,"tryCatchPattern":"try {\n  const { posts } = await collection(...);\n} catch (err) {\n  if (String(err.message).startsWith('twitter_collection_page_guard_hit')) {\n    // resume later using the oldest timestamp reached; raise maxPages for full backfill\n  } else throw err;\n}","preventionTips":["Set maxPages to cover the full walk from newest post to 'until'.","Do a one-time deep backfill, then frequent small incremental runs.","Raise page size to reduce required page count.","Track last collected timestamp to resume after guard hits.","Don't reuse small maxPages tuned for one account against larger ones."],"tags":["twitter","pagination","max-pages","completeness"],"backgroundTag":"pagination-incomplete","analyzedSha":"49907e53dc3ade5c223ff0c4c2c2785687cec4e6","analyzedAt":"2026-08-29T08:14:47.543Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}