{"record":{"id":"06a29e47861a7379","repo":"tonhowtf/omniget","slug":"server-returned-html-instead-of-media-url-may-have-expired","errorCode":null,"errorMessage":"Server returned HTML instead of media — URL may have expired","messagePattern":"Server returned HTML instead of media — URL may have expired","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src-tauri/omniget-core/src/core/direct_downloader.rs","lineNumber":513,"sourceCode":"\n    let mut offset = 0u64;\n    if existing_bytes > 0 {\n        if response.status() == reqwest::StatusCode::PARTIAL_CONTENT {\n            offset = existing_bytes;\n        } else if response.status() == reqwest::StatusCode::RANGE_NOT_SATISFIABLE {\n            let _ = std::fs::remove_file(part_path);\n            return Err(anyhow!(\"Range not satisfiable, restarting\"));\n        } else if !response.status().is_success() {\n            return Err(anyhow!(\"HTTP {} downloading {}\", response.status(), url));\n        }\n    } else if !response.status().is_success() {\n        return Err(anyhow!(\"HTTP {} downloading {}\", response.status(), url));\n    }\n\n    if let Some(ct) = response.headers().get(\"content-type\") {\n        if let Ok(ct_str) = ct.to_str() {\n            if ct_str.contains(\"text/html\") {\n                return Err(anyhow!(\n                    \"Server returned HTML instead of media — URL may have expired\"\n                ));\n            }\n        }\n    }\n\n    use std::io::Write;\n    let raw_file = if offset > 0 {\n        std::fs::OpenOptions::new().append(true).open(part_path)?\n    } else {\n        std::fs::File::create(part_path)?\n    };\n\n    let mut file = std::io::BufWriter::with_capacity(256 * 1024, raw_file);\n    let mut downloaded = offset;\n    let mut stream = response.bytes_stream();\n\n    let mut last_emit = std::time::Instant::now();","sourceCodeStart":495,"sourceCodeEnd":531,"githubUrl":"https://github.com/tonhowtf/omniget/blob/8600b91f4246848bac346874daa9e61c1fc5677a/src-tauri/omniget-core/src/core/direct_downloader.rs#L495-L531","documentation":"After a successful HTTP status, download_single_stream inspects the Content-Type header; if the origin responds with text/html, the 'media' URL actually served a web page (login page, error page, or redirect-to-home), so the downloader refuses to write HTML bytes into the target file. This protects users from saving corrupt non-media files.","triggerScenarios":"The remote URL returned 200 but with Content-Type containing 'text/html': typically an expired signed URL redirected to an HTML error/login page, a CDN served an interstitial/captcha page, or an extractor produced a stale or wrong media URL.","commonSituations":"YouTube/social-media direct URLs that expire within hours; paywalled or region-locked content where the origin answers with an HTML login page; Cloudflare/captcha challenge pages; hotlink protection redirecting to an HTML notice.","solutions":["Re-extract/refresh the media URL (signed URLs expire; fetch a fresh one) and retry","Add/refresh authentication cookies or headers so the origin stops serving a login page","Verify the URL with a HEAD request checking Content-Type starts with the expected media type before committing to a full download","Check for bot-protection (Cloudflare) and use appropriate headers or a resolver step","If it persists, confirm the URL points to the actual file, not a watch/landing page"],"exampleFix":"// before: trust any 200 response\nlet resp = client.get(url).send().await?;\n// after: pre-flight content-type check\nlet resp = client.get(url).send().await?;\nlet ct = resp.headers().get(\"content-type\").and_then(|v| v.to_str().ok()).unwrap_or(\"\");\nif ct.contains(\"text/html\") {\n    let fresh = reextract_media_url(&source).await?;\n    return download(fresh).await; // retry with a newly extracted URL\n}","handlingStrategy":"validation","validationCode":"let resp = client.head(url).send().await?;\nlet ct = resp.headers().get(\"content-type\").and_then(|v| v.to_str().ok()).unwrap_or(\"\");\nif ct.contains(\"text/html\") {\n    eprintln!(\"URL serves HTML, refresh the media URL before downloading\");\n}","typeGuard":"fn is_media_response(resp: &reqwest::Response) -> bool {\n    resp.headers().get(\"content-type\")\n        .and_then(|v| v.to_str().ok())\n        .map(|ct| !ct.contains(\"text/html\"))\n        .unwrap_or(true)\n}","tryCatchPattern":"match download(url, path).await {\n    Err(e) if e.to_string().contains(\"HTML instead of media\") => {\n        let fresh = reextract_media_url(&source).await?;\n        download(&fresh, path).await?;\n    }\n    r => r?,\n}","preventionTips":["Re-extract direct media URLs before each download — they often expire within minutes/hours","Validate Content-Type with a HEAD request before committing to a large transfer","Carry valid session cookies so origins don't answer with login pages","Detect redirect chains landing on HTML pages and abort early"],"tags":["network","http","content-type","expired-url","download"],"backgroundTag":"unexpected-response-shape","analyzedSha":"8600b91f4246848bac346874daa9e61c1fc5677a","analyzedAt":"2026-09-12T14:29:19.317Z","contentChangedAt":"2026-09-12T14:29:19.317Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}