{"record":{"id":"43238454623ab531","repo":"tonhowtf/omniget","slug":"no-photo-urls-found-in-html-mod","errorCode":null,"errorMessage":"No photo URLs found in HTML","messagePattern":"No photo URLs found in HTML","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"src-tauri/src/platforms/twitter/mod.rs","lineNumber":1053,"sourceCode":"\n    async fn request_html_media(&self, url: &str) -> anyhow::Result<Vec<serde_json::Value>> {\n        let mut request = self\n            .client\n            .get(url)\n            .header(\"User-Agent\", USER_AGENT)\n            .header(\"Accept-Language\", \"en\")\n            .header(\"Referer\", \"https://x.com/\");\n        if let Some(cookie) = Self::auth_cookie_string() {\n            request = request.header(\"Cookie\", cookie);\n        }\n        let response = request.send().await?;\n        if !response.status().is_success() {\n            return Err(anyhow!(\"HTML request returned HTTP {}\", response.status()));\n        }\n        let html = response.text().await?;\n        let items = Self::extract_html_photo_items(&html);\n        if items.is_empty() {\n            return Err(anyhow!(\"No photo URLs found in HTML\"));\n        }\n        tracing::debug!(\"[twitter] html extracted {} photo entries\", items.len());\n        Ok(items\n            .into_iter()\n            .map(|item| {\n                serde_json::json!({\n                    \"type\": \"photo\",\n                    \"media_url_https\": item.url,\n                })\n            })\n            .collect())\n    }\n\n    async fn try_graphql(&self, tweet_id: &str) -> anyhow::Result<Vec<serde_json::Value>> {\n        let token = self.get_guest_token(false).await?;\n\n        match self.request_tweet(tweet_id, &token).await {\n            Ok(json) => Self::extract_graphql_media(&json, tweet_id),","sourceCodeStart":1035,"sourceCodeEnd":1071,"githubUrl":"https://github.com/tonhowtf/omniget/blob/8600b91f4246848bac346874daa9e61c1fc5677a/src-tauri/src/platforms/twitter/mod.rs#L1035-L1071","documentation":"Thrown by TwitterDownloader::request_html_media when the tweet page was fetched successfully (2xx) but extract_html_photo_items found zero pbs.twimg.com/media URLs matching its regex. The HTML response simply did not contain recognizable photo media.","triggerScenarios":"Calling get_media_info on a tweet whose page HTML contains no pbs.twimg.com/media URLs: the tweet has video/GIF only (no photos), the tweet is a login wall rendering no media markup, or X changed its HTML structure so the regex no longer matches.","commonSituations":"Trying to download video-only tweets via the photo-oriented HTML scraper; X A/B-testing new markup; heavily JavaScript-rendered pages returning an app shell without media; login-required responses returning a 200 page with a sign-in wall.","solutions":["Confirm the tweet actually contains photos (this fallback only extracts photo URLs, not videos).","Check the saved HTML (log a snippet) for pbs.twimg.com/media occurrences; if the markup changed, update the regex in extract_html_photo_items.","Use the GraphQL or syndication strategies instead, or a yt-dlp fallback for video tweets.","Ensure the auth cookie is set so the page is not a login wall."],"exampleFix":"// before\nlet items = Self::extract_html_photo_items(&html);\nif items.is_empty() {\n    return Err(anyhow!(\"No photo URLs found in HTML\"));\n}\n// after: include diagnostics in the error\nlet items = Self::extract_html_photo_items(&html);\nif items.is_empty() {\n    if html.contains(\"pbs.twimg.com\") {\n        return Err(anyhow!(\"No photo URLs found in HTML (found twimg URLs but regex missed them)\"));\n    }\n    return Err(anyhow!(\"No photo URLs found in HTML (page likely login-walled or video-only)\"));\n}","handlingStrategy":"validation","validationCode":"// cheap pre-check that the page actually contains photo media markup\nlet html = fetch_page(&url).await?;\nif !html.contains(\"pbs.twimg.com/media\") {\n    eprintln!(\"tweet page has no photo media markup — photos-only fallback will fail\");\n}","typeGuard":"fn has_photo_markup(html: &str) -> bool {\n    html.contains(\"pbs.twimg.com/media\")\n}","tryCatchPattern":"match request_html_media(&url).await {\n    Err(e) if e.to_string().contains(\"No photo URLs found\") => {\n        // tweet is likely video-only or login-walled; switch strategy\n        let info = ytdlp_fallback(&url).await?;\n        Ok(info)\n    }\n    other => other,\n}","preventionTips":["Only use the HTML-photo fallback for tweets known to contain photos.","Check for a login wall (sign-in markers in HTML) before concluding there is no media.","Log a snippet of the HTML when extraction fails to spot markup changes early.","Keep the pbs.twimg.com regex updated against X markup changes.","Prefer GraphQL/syndication strategies; treat HTML as last resort only."],"tags":["scraping","parsing","twitter","html"],"backgroundTag":"empty-result-set","analyzedSha":"8600b91f4246848bac346874daa9e61c1fc5677a","analyzedAt":"2026-09-12T14:29:19.317Z","contentChangedAt":"2026-09-12T14:29:19.317Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}