{"record":{"id":"33fe805e92162206","repo":"jackwener/OpenCLI","slug":"could-not-parse-profile-data","errorCode":null,"errorMessage":"Could not parse profile data","messagePattern":"Could not parse profile data","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"clis/tiktok/profile.js","lineNumber":34,"sourceCode":"    columns: [\n        'username',\n        'name',\n        'followers',\n        'following',\n        'likes',\n        'videos',\n        'verified',\n        'bio',\n    ],\n    pipeline: [\n        { navigate: { url: 'https://www.tiktok.com/explore', settleMs: 5000 } },\n        { evaluate: `(async () => {\n  const username = \\${{ args.username | json }};\n  const res = await fetch('https://www.tiktok.com/@' + encodeURIComponent(username), { credentials: 'include' });\n  if (!res.ok) throw new Error('User not found: ' + username);\n  const html = await res.text();\n  const idx = html.indexOf('__UNIVERSAL_DATA_FOR_REHYDRATION__');\n  if (idx === -1) throw new Error('Could not parse profile data');\n  const start = html.indexOf('>', idx) + 1;\n  const end = html.indexOf('</script>', start);\n  const data = JSON.parse(html.substring(start, end));\n  const ud = data['__DEFAULT_SCOPE__'] && data['__DEFAULT_SCOPE__']['webapp.user-detail'];\n  const u = ud && ud.userInfo && ud.userInfo.user;\n  const s = ud && ud.userInfo && ud.userInfo.stats;\n  if (!u) throw new Error('User not found: ' + username);\n  return [{\n    username: u.uniqueId || username,\n    name: u.nickname || '',\n    bio: (u.signature || '').replace(/\\\\n/g, ' ').substring(0, 120),\n    followers: s && s.followerCount || 0,\n    following: s && s.followingCount || 0,\n    likes: s && s.heartCount || 0,\n    videos: s && s.videoCount || 0,\n    verified: u.verified ? 'Yes' : 'No',\n  }];\n})()","sourceCodeStart":16,"sourceCodeEnd":52,"githubUrl":"https://github.com/jackwener/OpenCLI/blob/49907e53dc3ade5c223ff0c4c2c2785687cec4e6/clis/tiktok/profile.js#L16-L52","documentation":"After fetching the profile HTML successfully, the script searches for the __UNIVERSAL_DATA_FOR_REHYDRATION__ script tag that carries TikTok's JSON state. If the marker string is absent (indexOf returns -1) it throws 'Could not parse profile data'. This means the page loaded but did not contain the expected embedded data blob.","triggerScenarios":"The fetched HTML for tiktok.com/@<username> lacks the __UNIVERSAL_DATA_FOR_REHYDRATION__ script — typically a login wall, captcha/bot-check page, consent/redirect interstitial, or a TikTok markup change removed/renamed the global data key.","commonSituations":"Datacenter IP or headless browser flagged by TikTok; cookie-consent banner intercepting; TikTok renaming the rehydration global in a frontend deploy; response being an SPA shell without SSR data.","solutions":["Retry with a logged-in, warmed browser session and residential IP to bypass bot/consent interstitials","Capture and inspect the returned HTML to see what page TikTok actually served","Check the library for updates — the marker may have been renamed by TikTok and selectors patched","Fall back to another data source or the mobile API for profile info"],"exampleFix":"// before\nconst data = await getProfile(username); // throws Could not parse profile data\n// after\ntry {\n  const data = await getProfile(username);\n} catch (e) {\n  if (/Could not parse profile data/.test(e.message)) {\n    console.error('TikTok page had no rehydration data (likely bot wall); retry with warm session');\n  } else throw e;\n}","handlingStrategy":"retry","validationCode":"const html = await (await fetch('https://www.tiktok.com/@' + username, { credentials: 'include' })).text();\nif (!html.includes('__UNIVERSAL_DATA_FOR_REHYDRATION__')) {\n  throw new Error('TikTok served a page without profile data (bot wall or markup change)');\n}","typeGuard":"function hasRehydrationData(html) {\n  return typeof html === 'string' && html.indexOf('__UNIVERSAL_DATA_FOR_REHYDRATION__') !== -1;\n}","tryCatchPattern":"let profile;\nfor (let attempt = 0; attempt < 3; attempt++) {\n  try {\n    profile = await run('tiktok profile', { username });\n    break;\n  } catch (e) {\n    if (!/Could not parse profile data/.test(e.message) || attempt === 2) throw e;\n    await new Promise(r => setTimeout(r, 5000 * (attempt + 1)));\n  }\n}","preventionTips":["Use a warmed, logged-in browser session with residential IPs to avoid consent/bot interstitials","Capture the failing HTML periodically to detect TikTok markup changes early","Keep the library updated for rehydration-key changes","Fail fast and alert when the served page is not a profile page (login wall/captcha detection)"],"tags":["tiktok","scraping","parse-error","dom-change","bot-detection"],"backgroundTag":"dom-structure-change","analyzedSha":"49907e53dc3ade5c223ff0c4c2c2785687cec4e6","analyzedAt":"2026-08-29T08:14:47.543Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}