{"record":{"id":"9ac4378e0e517a84","repo":"tonhowtf/omniget","slug":"http-arxiv","errorCode":null,"errorMessage":"HTTP {}","messagePattern":"HTTP \\{\\}","errorType":"http","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src-tauri/omniget-core/src/core/tools/arxiv.rs","lineNumber":1223,"sourceCode":"    };\n    report(&p, ID, \"done\", 3, Some(3), None);\n\n    Ok(ArxivDoc {\n        chars: markdown.chars().count() as u64,\n        math_blocks: count_math(&markdown),\n        meta,\n        markdown,\n        body_source: body_source.to_string(),\n        path,\n        source_files: files,\n        fallback_reason: fallback,\n    })\n}\n\nasync fn fetch_source(client: &reqwest::Client, r: &ArxivRef) -> anyhow::Result<SourceBundle> {\n    let resp = client.get(r.source_url()).send().await?;\n    if !resp.status().is_success() {\n        return Err(anyhow!(\"HTTP {}\", resp.status()));\n    }\n    let bytes = resp.bytes().await?;\n    extract_source(&bytes)\n}\n\nasync fn fetch_html(client: &reqwest::Client, r: &ArxivRef) -> anyhow::Result<String> {\n    let resp = client.get(r.html_url()).send().await?;\n    if !resp.status().is_success() {\n        return Err(anyhow!(\"HTTP {}\", resp.status()));\n    }\n    let html = resp.text().await?;\n    let html = mathml_to_tex(&html);\n    let md = htmd::convert(&html).map_err(|e| anyhow!(\"HTML para Markdown: {}\", e))?;\n    if md.trim().is_empty() {\n        return Err(anyhow!(\"pagina HTML vazia\"));\n    }\n    Ok(md)\n}","sourceCodeStart":1205,"sourceCodeEnd":1241,"githubUrl":"https://github.com/tonhowtf/omniget/blob/8600b91f4246848bac346874daa9e61c1fc5677a/src-tauri/omniget-core/src/core/tools/arxiv.rs#L1205-L1241","documentation":"arXiv fetch_source downloads the paper's LaTeX/e-print source bundle via reqwest and fails fast with anyhow!(\"HTTP {}\", status) whenever the response status is not a success (2xx). This is a deliberate early-exit so the caller never tries to extract a source archive from an error page. It surfaces the raw HTTP status code (e.g. 404, 503) as the error message.","triggerScenarios":"Calling fetch -> fetch_source with an ArxivRef whose source_url() returns a non-2xx response: arXiv ID does not exist, the paper has no e-print source (scanned PDF-only submissions), rate limiting (429/503 from arXiv), or network proxies returning 4xx/5xx.","commonSituations":"Typo in arXiv ID (404); arXiv API throttling during bulk fetching (503 with Retry-After); withdrawn papers; very old papers whose source is unavailable; corporate proxy blocking export.arxiv.org.","solutions":["Check the arXiv ID/ref is valid and the paper actually has an e-print source (test https://arxiv.org/e-print/<id> in a browser)","Implement retry with backoff for 429/503, honoring the Retry-After header","Log resp.status() and fall back to fetch_html or PDF download when source fetch fails","Verify network/proxy configuration allows reaching export.arxiv.org"],"exampleFix":"// before\nlet resp = client.get(r.source_url()).send().await?;\nif !resp.status().is_success() {\n    return Err(anyhow!(\"HTTP {}\", resp.status()));\n}\n// after\nlet resp = client.get(r.source_url()).send().await?;\nif resp.status() == reqwest::StatusCode::TOO_MANY_REQUESTS || resp.status() == reqwest::StatusCode::SERVICE_UNAVAILABLE {\n    tokio::time::sleep(Duration::from_secs(3)).await;\n    return fetch_source(client, r).await; // retry\n}\nif !resp.status().is_success() {\n    anyhow::bail!(\"arXiv source fetch failed ({}): {}\", resp.status(), r.source_url());\n}","handlingStrategy":"retry","validationCode":"// Rust: pre-check reachability of the e-print URL before the real call\nlet head = client.head(r.source_url()).send().await?;\nanyhow::ensure!(head.status().is_success(), \"arXiv source unavailable: {}\", head.status());","typeGuard":null,"tryCatchPattern":"match fetch_source(&client, &r).await {\n    Ok(bundle) => bundle,\n    Err(e) if e.to_string().contains(\"HTTP 5\") || e.to_string().contains(\"HTTP 429\") => {\n        tokio::time::sleep(Duration::from_secs(5)).await;\n        fetch_source(&client, &r).await?\n    }\n    Err(e) => return Err(e),\n}","preventionTips":["Validate arXiv IDs before fetching","Add exponential backoff for 429/503 from arXiv","Respect arXiv rate limits (delay between requests)","Always have a fallback path (HTML or PDF extraction)"],"tags":["http","network","arxiv","rust"],"backgroundTag":"http-error-response","analyzedSha":"8600b91f4246848bac346874daa9e61c1fc5677a","analyzedAt":"2026-09-12T14:29:19.317Z","contentChangedAt":"2026-09-12T14:29:19.317Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}