{"record":{"id":"c264b3d40e5eca9e","repo":"jackwener/OpenCLI","slug":"twitter-collection-request-error","errorCode":"twitter_collection_request_error","errorMessage":"twitter_collection_request_error: UserTweets returned ${payload.error}","messagePattern":"twitter_collection_request_error: UserTweets returned (.+?)","errorType":"error_code","errorClass":"CommandExecutionError","httpStatus":null,"severity":"error","filePath":"clis/twitter/collection.js","lineNumber":265,"sourceCode":"        completed: true,\n        stop_reason: stopReason,\n        requested_until: until.toISOString(),\n        pages_fetched: pagesFetched,\n        oldest_seen_at: oldestSeenAt ? oldestSeenAt.toISOString() : null,\n    };\n}\n\nasync function paginateCollection({ until, limit, maxPages = MAX_USER_TWEETS_PAGES, fetchPage, wait }) {\n    const seen = new Set();\n    const seenCursors = new Set();\n    const posts = [];\n    let cursor = null;\n    let oldestSeenAt = null;\n    for (let pageIndex = 0; pageIndex < maxPages; pageIndex++) {\n        if (pageIndex > 0 && wait) await wait();\n        const payload = await fetchPage(cursor, USER_TWEETS_PAGE_SIZE);\n        if (payload?.error) {\n            throw new CommandExecutionError(`twitter_collection_request_error: UserTweets returned ${payload.error}`);\n        }\n        const [pagePosts, nextCursor] = parseCollectionPage(payload, seen);\n        for (const post of pagePosts) {\n            if (posts.length >= limit) {\n                throw new CommandExecutionError('twitter_collection_limit_reached: pagination cannot prove completion');\n            }\n            const createdAt = parseCreatedAt(post);\n            if (!oldestSeenAt || createdAt < oldestSeenAt) oldestSeenAt = createdAt;\n            posts.push(post);\n            if (createdAt <= until) {\n                return {\n                    posts,\n                    receipt: completedReceipt('time_boundary_reached', until, pageIndex + 1, oldestSeenAt),\n                };\n            }\n        }\n        if (!nextCursor) {\n            return {","sourceCodeStart":247,"sourceCodeEnd":283,"githubUrl":"https://github.com/jackwener/OpenCLI/blob/49907e53dc3ade5c223ff0c4c2c2785687cec4e6/clis/twitter/collection.js#L247-L283","documentation":"paginateCollection checks each UserTweets payload for an error property before parsing. If the GraphQL response carries an error (auth failure, rate limit, bad cursor, etc.), the library surfaces it as twitter_collection_request_error including the API's own error text, aborting pagination since no completeness can be proven from an errored page.","triggerScenarios":"fetchPage returns a payload with truthy .error — X GraphQL errors such as rate limiting (429/TooManyRequests), expired auth cookies, invalid queryId after an X deploy, or cursor invalidated by account changes.","commonSituations":"Long-running paginations exceeding X rate limits; stale queryId in the bundled GraphQL request after X ships a new frontend build; expired guest/session token mid-run; network proxy injecting error JSON.","solutions":["Read payload.error in the message and address the specific X error (rate limit → back off and retry later; auth → refresh cookies/session).","Refresh authentication before long collection runs to avoid mid-pagination token expiry.","Update the library's queryId/GraphQL operation hashes if X deployed a new frontend version.","Add exponential backoff between pages instead of a fixed wait.","Reduce maxPages/page size to stay under rate limits for large accounts."],"exampleFix":"// before\nfor (let pageIndex = 0; pageIndex < maxPages; pageIndex++) {\n  if (pageIndex > 0 && wait) await wait();\n// after\nfor (let pageIndex = 0; pageIndex < maxPages; pageIndex++) {\n  if (pageIndex > 0 && wait) await wait(Math.min(baseDelay * 2 ** pageIndex, cap)); // backoff against rate limits","handlingStrategy":"retry","validationCode":"if (payload?.error) {\n  console.error('X GraphQL error before pagination:', payload.error);\n  return;\n}","typeGuard":"function isCleanPayload(payload) {\n  return payload != null && typeof payload === 'object' && !('error' in payload);\n}","tryCatchPattern":"try {\n  await paginateCollection(...);\n} catch (err) {\n  if (String(err.message).startsWith('twitter_collection_request_error')) {\n    await backoffRetry(maxAttempts, baseDelayMs); // handle rate limit / auth errors\n  } else throw err;\n}","preventionTips":["Check payload.error before each page and surface the API error text.","Use exponential backoff between pages to avoid 429 rate limits.","Refresh auth cookies before long collection runs.","Keep queryId/operation hashes current after X frontend deploys.","Cap maxPages/page size for large accounts to stay within limits."],"tags":["twitter","graphql","api-error","rate-limit"],"backgroundTag":"api-request-failed","analyzedSha":"49907e53dc3ade5c223ff0c4c2c2785687cec4e6","analyzedAt":"2026-08-29T08:14:47.543Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}