{"record":{"id":"0de14334f03dfd66","repo":"jackwener/OpenCLI","slug":"twitter-device-follow-returned-non-json-response","errorCode":null,"errorMessage":"Twitter device-follow returned non-JSON response: ${data.detail || 'unknown parse error'}","messagePattern":"Twitter device-follow returned non-JSON response: (.+?)","errorType":"exception","errorClass":"CommandExecutionError","httpStatus":null,"severity":"error","filePath":"clis/twitter/device-follow.js","lineNumber":160,"sourceCode":"            'X-Csrf-Token': ct0,\n            'X-Twitter-Auth-Type': 'OAuth2Session',\n            'X-Twitter-Active-User': 'yes',\n        });\n        const data = await page.evaluate(`async () => {\n        try {\n          const r = await fetch(\"${apiUrl}\", { method: \"GET\", headers: ${headers}, credentials: 'include' });\n          if (!r.ok) return { error: r.status };\n          try {\n            return await r.json();\n          } catch (e) {\n            return { errorKind: 'non_json', detail: String(e && e.message || e) };\n          }\n        } catch (e) {\n          return { errorKind: 'exception', detail: String(e && e.message || e) };\n        }\n      }`);\n        if (data?.errorKind === 'non_json') {\n            throw new CommandExecutionError(`Twitter device-follow returned non-JSON response: ${data.detail || 'unknown parse error'}`);\n        }\n        if (data?.errorKind === 'exception') {\n            throw new CommandExecutionError(`Twitter device-follow fetch failed: ${data.detail || 'unknown error'}`);\n        }\n        if (data?.error) {\n            if (data.error === 401 || data.error === 403) {\n                throw new AuthRequiredError('x.com', `Twitter device-follow returned HTTP ${data.error}`);\n            }\n            throw new CommandExecutionError(describeTwitterApiError('device_follow', data.error));\n        }\n        const parsed = parseDeviceFollow(data, new Set());\n        if (!parsed) {\n            throw new CommandExecutionError('Twitter device-follow response was missing the expected timeline/globalObjects shape.');\n        }\n        if (parsed.malformedEntries > 0 || parsed.unmatchedTweetEntries > 0) {\n            throw new CommandExecutionError('Twitter device-follow entries could not be joined to tweet/user objects.');\n        }\n        if (parsed.rows.length === 0) {","sourceCodeStart":142,"sourceCodeEnd":178,"githubUrl":"https://github.com/jackwener/OpenCLI/blob/49907e53dc3ade5c223ff0c4c2c2785687cec4e6/clis/twitter/device-follow.js#L142-L178","documentation":"The in-page fetch of the device-follow GraphQL endpoint wraps response parsing; when the body cannot be parsed as JSON it reports errorKind 'non_json' with a detail string, which the Node side rethrows as a CommandExecutionError. This indicates the endpoint answered, but not with the expected JSON payload.","triggerScenarios":"x.com returning an HTML error/interstitial page (rate-limit wall, login wall, Cloudflare challenge), a network error page, or an empty/redirect response for the device_follow GraphQL request inside the page context.","commonSituations":"Heavy scraping triggering rate limits or bot-detection interstitials; logged-out or suspicious sessions being redirected to HTML; temporary x.com outages returning error pages instead of GraphQL JSON.","solutions":["Re-run after a delay to rule out a transient rate-limit/interstitial page","Confirm the session is authenticated (ct0 cookie present) so x.com does not redirect to a login wall","Retry from a different IP / clear cookies to escape bot-detection challenges","Inspect data.detail in the message to identify the specific parse failure"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// No pre-call check possible; verify session first to reduce login-wall risk\nconst cookies = await page.getCookies({ url: 'https://x.com' });\nif (!cookies.some((c) => c.name === 'ct0')) throw new Error('log in first');","typeGuard":"function isJsonObject(text) {\n  try { const v = JSON.parse(text); return v !== null && typeof v === 'object'; } catch { return false; }\n}","tryCatchPattern":"try {\n  await cli.twitter.deviceFollow();\n} catch (e) {\n  if (e instanceof CommandExecutionError && /non-JSON response/.test(e.message)) {\n    await sleep(30_000); // back off: likely rate-limit/interstitial page\n    return retryOnce();\n  }\n  throw e;\n}","preventionTips":["Throttle request frequency to avoid x.com bot-detection interstitials","Keep the session authenticated so x.com doesn't serve HTML login walls","Retry with exponential backoff on non-JSON responses","Use a clean profile without ad-blocker extensions that can intercept fetches"],"tags":["network","json","scraping","rate-limit"],"backgroundTag":"non-json-response","analyzedSha":"49907e53dc3ade5c223ff0c4c2c2785687cec4e6","analyzedAt":"2026-08-29T08:14:47.543Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}