{"record":{"id":"620bc7303c4d51a4","repo":"tonhowtf/omniget","slug":"could-not-extract-data-from-embed","errorCode":null,"errorMessage":"Could not extract data from embed","messagePattern":"Could not extract data from embed","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src-tauri/omniget-core/src/platforms/instagram.rs","lineNumber":479,"sourceCode":"\n        if let Some(json_str) = Self::regex_extract(r#\"\"init\",\\[\\],\\[(.*?)\\]\\],\"#, &html) {\n            if let Ok(embed_data) = serde_json::from_str::<serde_json::Value>(&json_str) {\n                if let Some(context_json) = embed_data.get(\"contextJSON\").and_then(|v| v.as_str()) {\n                    let context: serde_json::Value = serde_json::from_str(context_json)?;\n                    return Ok(context);\n                }\n            }\n        }\n\n        if let Some(json_str) = Self::regex_extract(\n            r#\"window\\.__additionalDataLoaded\\('extra',\\s*(\\{.*?\\})\\s*\\)\"#,\n            &html,\n        ) {\n            let data: serde_json::Value = serde_json::from_str(&json_str)?;\n            return Ok(data);\n        }\n\n        Err(anyhow!(\"Could not extract data from embed\"))\n    }\n\n    async fn fallback_ytdlp(&self, url: &str, post_id: &str) -> anyhow::Result<MediaInfo> {\n        let ytdlp_path = crate::core::ytdlp::ensure_ytdlp().await?;\n        let json = crate::core::ytdlp::get_video_info(&ytdlp_path, url, &[]).await?;\n        let mut info =\n            crate::platforms::generic_ytdlp::GenericYtdlpDownloader::parse_video_info(&json)?;\n\n        info.title = format!(\"instagram_{}\", post_id);\n        info.platform = \"instagram\".to_string();\n\n        let post_url = format!(\"https://www.instagram.com/p/{}/\", post_id);\n        for q in &mut info.available_qualities {\n            q.format = \"ytdlp\".to_string();\n            q.url = post_url.clone();\n        }\n\n        Ok(info)","sourceCodeStart":461,"sourceCodeEnd":497,"githubUrl":"https://github.com/tonhowtf/omniget/blob/8600b91f4246848bac346874daa9e61c1fc5677a/src-tauri/omniget-core/src/platforms/instagram.rs#L461-L497","documentation":"request_embed fetches the embed HTML page (https://www.instagram.com/p/{id}/embed/captioned/) and tries to scrape an embedded JSON payload via two regexes (the \"init\",[]... contextJSON blob or window.__additionalDataLoaded('extra', ...)). This error is thrown when neither regex matched the returned HTML, meaning Instagram served an embed page without extractable JSON data.","triggerScenarios":"Calling get_media_info whose post_id leads to an embed page lacking both \"init\",[],[...] contextJSON and window.__additionalDataLoaded('extra', {...}) markers — typically when Instagram serves a login/consent wall, an error page, or changed its embed HTML template.","commonSituations":"Instagram A/B-tested or changed the embed page markup so the regexes no longer match; embeds disabled for the post; rate-limited request returned a challenge page; post ID invalid so embed returns an error page.","solutions":["Inspect the raw embed HTML returned for the failing post_id and update the extraction regexes to the current markup.","Check whether the request was redirected to a login or challenge page and handle that case explicitly.","Confirm the post ID is valid by fetching /p/{id}/embed/ in a browser.","Let the pipeline fall through to fallback_ytdlp, and keep yt-dlp updated as the last-resort extractor.","Avoid calling request_embed for post types Instagram refuses to embed (e.g. private accounts)."],"exampleFix":null,"handlingStrategy":"fallback","validationCode":"// Confirm the post id is a plausible shortcode before hitting the embed page\nfn plausible_shortcode(id: &str) -> bool {\n    !id.is_empty() && id.len() <= 32 && id.chars().all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '-')\n}","typeGuard":null,"tryCatchPattern":"match request_embed(&post_id).await {\n    Ok(data) => data,\n    Err(e) if e.to_string().contains(\"Could not extract data from embed\") => {\n        // Instagram changed embed markup or served a challenge page; use yt-dlp fallback\n        return fallback_ytdlp(url, &post_id).await;\n    }\n    Err(e) => return Err(e),\n}","preventionTips":["Keep yt-dlp fallback available and updated","Capture and review raw embed HTML when the regexes fail","Detect login/challenge redirects explicitly","Don't retry the embed blindly — markup changes are persistent, not transient"],"tags":["instagram","scraping","html-parsing","regex"],"backgroundTag":"unexpected-response-shape","analyzedSha":"8600b91f4246848bac346874daa9e61c1fc5677a","analyzedAt":"2026-09-12T14:29:19.317Z","contentChangedAt":"2026-09-12T14:29:19.317Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}