{"record":{"id":"1180e4ddc3831b4c","repo":"tonhowtf/omniget","slug":"http-media","errorCode":null,"errorMessage":"HTTP {}","messagePattern":"HTTP \\{\\}","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src-tauri/omniget-core/src/core/tools/x/media.rs","lineNumber":137,"sourceCode":"            tokio::time::sleep(std::time::Duration::from_millis(250)).await;\n        }\n    }\n    super::report(progress, job, \"done\", done as u64, Some(total as u64), None);\n    Ok(result)\n}\n\nasync fn fetch_to(\n    client: &reqwest::Client,\n    url: &str,\n    path: &std::path::Path,\n) -> anyhow::Result<()> {\n    let resp = client\n        .get(url)\n        .header(\"Referer\", \"https://x.com/\")\n        .send()\n        .await?;\n    if !resp.status().is_success() {\n        return Err(anyhow!(\"HTTP {}\", resp.status()));\n    }\n    let bytes = resp.bytes().await?;\n    let part = path.with_extension(\"part\");\n    tokio::fs::write(&part, &bytes).await?;\n    tokio::fs::rename(&part, path).await?;\n    Ok(())\n}\n\n/// Todas as midias publicas de um perfil (aba Midia), ate `limit` posts.\npub async fn download_profile(\n    input: &str,\n    dest: &str,\n    limit: usize,\n    photos: bool,\n    videos: bool,\n    progress: ProgressFn,\n) -> anyhow::Result<MediaResult> {\n    let handle = super::handle_from(input)","sourceCodeStart":119,"sourceCodeEnd":155,"githubUrl":"https://github.com/tonhowtf/omniget/blob/8600b91f4246848bac346874daa9e61c1fc5677a/src-tauri/omniget-core/src/core/tools/x/media.rs#L119-L155","documentation":"fetch_to downloads a single media file from x.com CDN and throws `HTTP {}` when the GET response status is not a success (2xx). The library checks `resp.status().is_success()` after sending the request and aborts early, before writing the .part file, so no partial download is left on disk. The status code is embedded in the message (e.g. `HTTP 404 Not Found`).","triggerScenarios":"Calling download_posts which calls fetch_to; the reqwest GET (with Referer header https://x.com/) to the media URL returns 403/404/410/5xx. Typical causes: the media URL expired (signed CDN URLs from fxtwitter/GraphQL expire), the tweet/media was deleted, or the CDN rejected the request for missing auth/rate limiting.","commonSituations":"Downloading media from old or deleted tweets whose pbs.twimg.com URLs returned 410 Gone; expired signed URLs after the tweet JSON was fetched minutes earlier; Twitter CDN throttling with 429; corporate proxy returning 403.","solutions":["Re-fetch the tweet metadata (fxtwitter or GraphQL) to get a fresh media URL, then retry the download immediately after fetching","Retry with backoff on 429/5xx statuses; treat 404/410 as permanent and skip that media item","Ensure the Referer/UA headers are preserved (some CDN paths reject bare requests); log resp.status() to distinguish permanent vs transient failures"],"exampleFix":"// before\nif !resp.status().is_success() {\n    return Err(anyhow!(\"HTTP {}\", resp.status()));\n}\n// after\nif resp.status() == reqwest::StatusCode::TOO_MANY_REQUESTS || resp.status().is_server_error() {\n    tokio::time::sleep(std::time::Duration::from_secs(2)).await;\n    // retry the request once before giving up\n}\nif !resp.status().is_success() {\n    anyhow::bail!(\"HTTP {} for {}\", resp.status(), url);\n}","handlingStrategy":"retry","validationCode":"// optionally pre-check reachability (rarely possible for signed URLs)\nlet head = client.head(media_url).send().await?;\nif !head.status().is_success() {\n    eprintln!(\"midia indisponivel: {}\", head.status()); // skip before calling download path\n}","typeGuard":"fn is_retryable(status: reqwest::StatusCode) -> bool {\n    status == reqwest::StatusCode::TOO_MANY_REQUESTS || status.is_server_error()\n}","tryCatchPattern":"match fetch_media(url, path).await {\n    Err(e) if e.to_string().starts_with(\"HTTP 429\") || e.to_string().starts_with(\"HTTP 5\") => {\n        tokio::time::sleep(Duration::from_secs(5)).await; // retry once\n    }\n    Err(e) if e.to_string().starts_with(\"HTTP 40\") => skip_item(e), // permanent: skip\n    Err(e) => return Err(e),\n    Ok(()) => {}\n}","preventionTips":["Download media immediately after fetching tweet metadata, before signed CDN URLs expire","Treat 404/410 as permanent and skip; retry only 429/5xx with backoff","Always send Referer/UA headers matching the library's request","Keep the atomic .part+rename pattern so failed downloads never leave corrupt files"],"tags":["network","http","download","rate-limit"],"backgroundTag":"http-error-response","analyzedSha":"8600b91f4246848bac346874daa9e61c1fc5677a","analyzedAt":"2026-09-12T14:29:19.317Z","contentChangedAt":"2026-09-12T14:29:19.317Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}