{"record":{"id":"4ffddc470c3c99b0","repo":"Zie619/n8n-workflows","slug":"could-not-find-universal-data-for-rehydration","errorCode":null,"errorMessage":"Could not find __UNIVERSAL_DATA_FOR_REHYDRATION__ script in the HTML.","messagePattern":"Could not find __UNIVERSAL_DATA_FOR_REHYDRATION__ script in the HTML\\.","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"workflows/Code/1802_Code_Manual_Import_Webhook.json","lineNumber":76,"sourceCode":"              \"name\": \"User-Agent\",\n              \"value\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/91.0.4472.124\"\n            }\n          ]\n        }\n      },\n      \"typeVersion\": 4.2,\n      \"notes\": \"This httpRequest node performs automated tasks as part of the workflow.\"\n    },\n    {\n      \"id\": \"734a5304-f67f-4ace-a1da-0d268664452c\",\n      \"name\": \"Scrape raw video URL\",\n      \"type\": \"n8n-nodes-base.code\",\n      \"position\": [\n        480,\n        20\n      ],\n      \"parameters\": {\n        \"jsCode\": \"const html = $input.first().json.data;\\nconst headers = $input.first().json.headers || {};\\nconst cookies = headers['set-cookie'] || [];\\n\\nif (!html) {\\n  throw new Error(\\\"HTML body is undefined. Check the previous node's output.\\\");\\n}\\nconst regex = /<script id=\\\"__UNIVERSAL_DATA_FOR_REHYDRATION__\\\" type=\\\"application\\\\/json\\\">([\\\\s\\\\S]*?)<\\\\/script>/;\\nconst match = html.match(regex);\\n\\nif (match) {\\n  const jsonStr = match[1];\\n  const data = JSON.parse(jsonStr);\\n  const videoUrl = data?.__DEFAULT_SCOPE__?.[\\\"webapp.video-detail\\\"]?.itemInfo?.itemStruct?.video?.playAddr;\\n  if (!videoUrl) {\\n    throw new Error(\\\"Could not find video URL in the JSON data.\\\");\\n  }\\n  return [{ json: { videoUrl, cookies: cookies.join('; ') } }];\\n} else {\\n  throw new Error(\\\"Could not find __UNIVERSAL_DATA_FOR_REHYDRATION__ script in the HTML.\\\");\\n}\"\n      },\n      \"typeVersion\": 2,\n      \"notes\": \"This code node performs automated tasks as part of the workflow.\"\n    },\n    {\n      \"id\": \"f574ccb8-6f5f-4e55-a2d5-7ad775d3c4e5\",\n      \"name\": \"Output video file without watermark\",\n      \"type\": \"n8n-nodes-base.httpRequest\",\n      \"position\": [\n        900,\n        20\n      ],\n      \"parameters\": {\n        \"url\": \"{{ $env.BASE_URL }}\",\n        \"options\": {\n          \"response\": {\n            \"response\": {\n              \"responseFormat\": \"file\"","sourceCodeStart":58,"sourceCodeEnd":94,"githubUrl":"https://github.com/Zie619/n8n-workflows/blob/94007c1445d9258a7da116646b79473e7c7c3282/workflows/Code/1802_Code_Manual_Import_Webhook.json#L58-L94","documentation":"The third and earliest scraping failure in the 'Scrape raw video URL' node: the regex looking for <script id=\"__UNIVERSAL_DATA_FOR_REHYDRATION__\" type=\"application/json\"> found no match in the fetched HTML at all. This means the page returned is not a standard TikTok video page (or the script tag markup changed).","triggerScenarios":"TikTok served a captcha/verify/bot-check interstitial, a login wall, a redirect page, or an error page instead of the video page; the upstream request went to a non-video URL (profile page, shortened link that wasn't resolved); TikTok changed the script tag's id, type attribute, or attribute order so the literal regex no longer matches.","commonSituations":"Scraping from server/datacenter IPs without browser headers; URL passed as a tiktok.com/@user short link that redirects to an HTML shell without hydration data; frontend deploy changing the hydration script id; n8n instance behind a proxy that strips or rewrites script tags.","solutions":["Log a slice of the fetched HTML (e.g. html.slice(0, 500)) to identify what page actually came back (captcha, login, error).","Add realistic browser headers (User-Agent, Accept-Language, Referer: https://www.tiktok.com/) and follow redirects on the upstream HTTP node.","Confirm the input URL is a direct video URL (contains /video/<id> or a valid vt.tv/tiktok.com short link that resolves).","If TikTok changed the marker, relax the regex to match on the id with any attribute order: /<script[^>]*id=\"__UNIVERSAL_DATA_FOR_REHYDRATION__\"[^>]*>([\\s\\S]*?)<\\/script>/."],"exampleFix":"// before\nconst regex = /<script id=\"__UNIVERSAL_DATA_FOR_REHYDRATION__\" type=\"application\\/json\">([\\s\\S]*?)<\\/script>/;\nconst match = html.match(regex);\n\n// after (attribute-order tolerant)\nconst regex = /<script[^>]*id=\"__UNIVERSAL_DATA_FOR_REHYDRATION__\"[^>]*>([\\s\\S]*?)<\\/script>/;\nconst match = html.match(regex);\nif (!match) {\n  throw new Error(`No hydration script found. Page starts with: ${html.slice(0, 200)}`);\n}","handlingStrategy":"fallback","validationCode":"const looksLikeVideoPage = /__UNIVERSAL_DATA_FOR_REHYDRATION__/.test(html) || /\"webapp.video-detail\"/.test(html);","typeGuard":"function extractHydrationJson(html) {\n  const m = html.match(/<script[^>]*id=\"__UNIVERSAL_DATA_FOR_REHYDRATION__\"[^>]*>([\\s\\S]*?)<\\/script>/);\n  return m ? m[1] : null;\n}","tryCatchPattern":"try { JSON.parse(jsonStr); } catch (e) { throw new Error(`Hydration JSON malformed: ${e.message}`); }","preventionTips":["Match the script by id with attribute-order-tolerant regex, not a literal tag string.","Detect bot/captcha pages early (title or keyword check) and retry with cookies/headers instead of parsing.","Include a snippet of the fetched HTML in the error for fast diagnosis.","Keep browser headers and redirect-following enabled on the fetch node."],"tags":["n8n","tiktok","web-scraping","bot-detection","regex"],"backgroundTag":null,"analyzedSha":"94007c1445d9258a7da116646b79473e7c7c3282","analyzedAt":"2026-08-15T04:10:37.591Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}