tonhowtf/omniget · error

No photo URLs found in HTML

Error message

No photo URLs found in HTML

What it means

In `request_html_media` (src-tauri/omniget-core/src/platforms/twitter.rs:986), after a successful HTTP fetch, the HTML is parsed with `extract_html_photo_items`; if it yields zero photo entries this error is thrown. It means the page was retrieved but the expected photo URL patterns could not be extracted — X.com's markup changed, the page is a login/consent wall, or the tweet genuinely has no photos.

Solutions

1. Confirm the URL actually points to a tweet containing photo media; text/video-only tweets will never yield photo URLs.
2. Log/dump the fetched HTML (tracing::debug) and update extract_html_photo_items' patterns to match the current X.com markup.
3. Send the auth cookie so X serves the full logged-in page instead of a login wall — check Self::auth_cookie_string() returns a valid session.
4. Set a realistic browser User-Agent alongside the Referer header so X doesn't serve a stripped shell page.
5. Fall back to the primary API/syndication path instead of HTML scraping if the extractor is stale.

Example fix

// before: silent empty result
let html = response.text().await?;
let items = Self::extract_html_photo_items(&html);
if items.is_empty() {
    return Err(anyhow!("No photo URLs found in HTML"));
}

// after: diagnose wall vs. real no-media
let html = response.text().await?;
if html.contains("log-in") || html.contains("Enter your password") {
    return Err(anyhow!("Twitter returned a login wall; auth cookie missing or expired"));
}
let items = Self::extract_html_photo_items(&html);
if items.is_empty() {
    tracing::warn!("[twitter] no photo entries; html_len={} (extractor may be stale)", html.len());
    return Err(anyhow!("No photo URLs found in HTML"));
}

// Prefetch and sanity-check that the tweet actually contains photo media
// (e.g. via the syndication API) before calling the HTML-scrape path:
let meta = client.get(format!("https://cdn.syndication.twimg.com/tweet-result?id={id}"))
    .send().await?.json::<serde_json::Value>().await?;
let has_photos = meta["mediaDetails"].as_array()
    .map(|a| a.iter().any(|m| m["type"] == "photo"))
    .unwrap_or(false);
if !has_photos {
    anyhow::bail!("tweet {} has no photo media; skip HTML scrape", id);
}

fn has_photo_entries(html: &str) -> bool {
    html.contains("pbs.twimg.com/media") // quick marker that photo URLs exist in the page
}

match native_get_media_info(url).await {
    Ok(info) => use_media(info),
    Err(e) if e.to_string().contains("No photo URLs found in HTML") => {
        // HTML extractor is stale or page was a login wall: fall back to API path
        match native_get_media_info_via_api(url).await {
            Ok(info) => use_media(info),
            Err(inner) => report("neither HTML scrape nor API returned media", inner),
        }
    }
    Err(e) => report("twitter fetch failed", e),
}

When it happens

Trigger: Calling native_get_media_info for a Twitter photo URL where the fetched HTML contains no extractable photo items: tweet is a video-only or text-only post, X returns a login wall/'Something went wrong' page, or DOM structure changed so extract_html_photo_items' selectors/regexes no longer match.

Common situations: X.com frontend redesign breaking the extractor's patterns, logged-out scraping hitting the guest/login interstitial, pointing the tool at a non-photo tweet URL, or Twitter serving a JS-shell page without embedded JSON photo data to the request's User-Agent.

Understand the failure class

Background: EmptyResultError / "no results found": when an API or scraper succeeds but returns zero rows — this error's family across 9 libraries.

Related errors

• No media found in tweet
• No photo URLs found in HTML
• empty stream url
• não achei nenhuma faixa em {}
• nenhum favorito foi lido: esses perfis costumam exigir a ses

AI-assisted analysis of tonhowtf/omniget@8600b91f42 (2026-09-12). Data as JSON: /api/errors/7672731d4308df4c. Report an issue: GitHub.

Appendix: source

Thrown at src-tauri/omniget-core/src/platforms/twitter.rs:986

    async fn request_html_media(&self, url: &str) -> anyhow::Result<Vec<serde_json::Value>> {
        let mut request = self
            .client
            .get(url)
            .header("User-Agent", USER_AGENT)
            .header("Accept-Language", "en")
            .header("Referer", "https://x.com/");
        if let Some(cookie) = Self::auth_cookie_string() {
            request = request.header("Cookie", cookie);
        }
        let response = request.send().await?;
        if !response.status().is_success() {
            return Err(anyhow!("HTML request returned HTTP {}", response.status()));
        }
        let html = response.text().await?;
        let items = Self::extract_html_photo_items(&html);
        if items.is_empty() {
            return Err(anyhow!("No photo URLs found in HTML"));
        }
        tracing::debug!("[twitter] html extracted {} photo entries", items.len());
        Ok(items
            .into_iter()
            .map(|item| {
                serde_json::json!({
                    "type": "photo",
                    "media_url_https": item.url,
                })
            })
            .collect())
    }

    async fn try_graphql(&self, tweet_id: &str) -> anyhow::Result<Vec<serde_json::Value>> {
        let token = self.get_guest_token(false).await?;

        match self.request_tweet(tweet_id, &token).await {
            Ok(json) => Self::extract_graphql_media(&json, tweet_id),

View on GitHub (pinned to 8600b91f42)