tonhowtf/omniget · error · anyhow::Error

Could not extract data from embed

Error message

Could not extract data from embed

What it means

Raised by `request_embed` in InstagramDownloader after every JSON extraction strategy over the fetched embed page (regex-extracted JSON blobs) failed to parse as serde_json::Value. It means Instagram returned an embed page that did not contain any recognizable serialized data payload (e.g. no `shortcode_media`/`contextJSON` blob), so no structured post data could be produced. This is the top-level sentinel for 'embed fetch succeeded HTTP-wise but content was unusable'.

Solutions

1. Retry the download through the built-in fallback path (`fallback_ytdlp`), which callers already use when `request_embed` fails
2. Verify the post is public and renders at https://www.instagram.com/p/<post_id>/embed/ in a plain browser
3. Check for a login-wall or consent page in the fetched HTML and route requests through a residential proxy or add valid cookies
4. Update the extraction regexes/this crate to match Instagram's current embed JSON shape

Example fix

// before
let media = self.request_embed(&post_id).await.ok();
// after
let media = match self.request_embed(&post_id).await {
    Ok(m) => m,
    Err(e) => {
        tracing::warn!("embed extraction failed: {e:#}; falling back to yt-dlp");
        return self.fallback_ytdlp(url, &post_id).await;
    }
};

// pre-check before relying on embed extraction
async fn embed_has_data(client: &reqwest::Client, post_id: &str) -> bool {
    let html = client.get(format!("https://www.instagram.com/p/{post_id}/embed/captioned/")).send().await.ok()?.text().await.ok()?;
    html.contains("contextJSON") || html.contains("shortcode_media")
}

fn has_extractable_json(html: &str) -> bool {
    html.contains("contextJSON") || html.contains("shortcode_media") || html.contains("display_url")
}

match downloader.request_embed(&post_id).await {
    Ok(data) => data,
    Err(e) if e.to_string().contains("Could not extract data from embed") => {
        // fall back to yt-dlp path
        downloader.fallback_ytdlp(url, &post_id).await?
    }
    Err(e) => return Err(e),
}

When it happens

Trigger: Calling `request_embed(post_id)` when Instagram's embed endpoint returns an HTML page without parseable JSON — e.g. rate-limited/login-wall HTML, a removed or private post, or Instagram changing the embedded JSON shape so all regex extraction patterns miss.

Common situations: Instagram layout/API change breaking extraction regexes; the post is private, deleted, or age-restricted; datacenter IP gets served a consent/login interstitial; a typo'd post_id produces an empty embed page.

Related errors

• Could not extract data from embed
• No media found in embed
• nao achei os bundles JS do X na pagina
• não achei o id da sua conta na página de importação do Goodr
• nao encontrei slides nessa pagina (o SlideShare pode ter mud

AI-assisted analysis of tonhowtf/omniget@8600b91f42 (2026-09-12). Data as JSON: /api/errors/7fbd9a94e1a66ebb. Report an issue: GitHub.

Appendix: source

Thrown at src-tauri/src/platforms/instagram/mod.rs:471

        if let Some(json_str) = Self::regex_extract(r#""init",\[\],\[(.*?)\]\],"#, &html) {
            if let Ok(embed_data) = serde_json::from_str::<serde_json::Value>(&json_str) {
                if let Some(context_json) = embed_data.get("contextJSON").and_then(|v| v.as_str()) {
                    let context: serde_json::Value = serde_json::from_str(context_json)?;
                    return Ok(context);
                }
            }
        }

        if let Some(json_str) = Self::regex_extract(
            r#"window\.__additionalDataLoaded\('extra',\s*(\{.*?\})\s*\)"#,
            &html,
        ) {
            let data: serde_json::Value = serde_json::from_str(&json_str)?;
            return Ok(data);
        }

        Err(anyhow!("Could not extract data from embed"))
    }

    async fn fallback_ytdlp(&self, url: &str, post_id: &str) -> anyhow::Result<MediaInfo> {
        let ytdlp_path = crate::core::ytdlp::ensure_ytdlp().await?;
        let json = crate::core::ytdlp::get_video_info(&ytdlp_path, url, &[]).await?;
        let mut info =
            crate::platforms::generic_ytdlp::GenericYtdlpDownloader::parse_video_info(&json)?;

        info.title = format!("instagram_{}", post_id);
        info.platform = "instagram".to_string();

        let post_url = format!("https://www.instagram.com/p/{}/", post_id);
        for q in &mut info.available_qualities {
            q.format = "ytdlp".to_string();
            q.url = post_url.clone();
        }

        Ok(info)

View on GitHub (pinned to 8600b91f42)