{"record":{"id":"a96412e5678b7eef","repo":"FlowiseAI/Flowise","slug":"failed-to-download-pdf-response-status-respo","errorCode":null,"errorMessage":"Failed to download PDF: ${response.status} ${response.statusText}","messagePattern":"Failed to download PDF: (.+?) (.+?)","errorType":"http","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/components/nodes/tools/Arxiv/core.ts","lineNumber":175,"sourceCode":"        const response = await fetch(url)\n        if (!response.ok) {\n            throw new Error(`Arxiv API error: ${response.status} ${response.statusText}`)\n        }\n\n        const xmlText = await response.text()\n        return this.parseArxivResponse(xmlText)\n    }\n\n    private async downloadAndExtractPdf(arxivId: string): Promise<string> {\n        // Extract clean arxiv ID from full URL if needed\n        const cleanId = arxivId.replace('http://arxiv.org/abs/', '').replace('https://arxiv.org/abs/', '')\n        const pdfUrl = `https://arxiv.org/pdf/${cleanId}.pdf`\n\n        this.logger?.info(`[${this.orgId}]: Downloading PDF from: ${pdfUrl}`)\n\n        const response = await fetch(pdfUrl)\n        if (!response.ok) {\n            throw new Error(`Failed to download PDF: ${response.status} ${response.statusText}`)\n        }\n\n        // Get PDF buffer and create blob\n        const buffer = await response.buffer()\n        const blob = new Blob([new Uint8Array(buffer)])\n\n        // Use PDFLoader to extract text (same as Pdf.ts)\n        const loader = new PDFLoader(blob, {\n            splitPages: false,\n            pdfjs: () =>\n                // @ts-ignore\n                this.legacyBuild ? import('pdfjs-dist/legacy/build/pdf.js') : import('pdf-parse/lib/pdf.js/v1.10.100/build/pdf.js')\n        })\n\n        const docs = await loader.load()\n        return docs.map((doc) => doc.pageContent).join('\\n')\n    }\n","sourceCodeStart":157,"sourceCodeEnd":193,"githubUrl":"https://github.com/FlowiseAI/Flowise/blob/abe4a8601a058047b350c260676826e21dd14101/packages/components/nodes/tools/Arxiv/core.ts#L157-L193","documentation":"Thrown by ArxivTool.downloadAndExtractPdf when the HTTP GET to https://arxiv.org/pdf/<id>.pdf returns a non-2xx status. Status and statusText are interpolated. PDF downloads fail for reasons distinct from the search API — paywalls do not apply, but rate limiting, withdrawn papers, and CDN errors do.","triggerScenarios":"Paper was withdrawn and the PDF endpoint returns 404; arxiv.org CDN returns 503 under load; 429 because PDF downloads are more aggressively rate-limited than search; the id contained a version suffix the URL builder did not expect.","commonSituations":"Bulk ingestion of many papers in a loop; old papers whose PDF was removed; running from a shared cloud IP that Arxiv throttles for PDF downloads.","solutions":["Retry on 429/5xx with backoff and reduce concurrency.","Verify the arxivId is a clean id (no leading URL, correct version format) before constructing pdfUrl.","If the paper is withdrawn, treat 404 as a soft skip and fall back to the abstract summary (set continueOnFailure=true).","Cache downloaded PDFs to avoid re-fetching across runs."],"exampleFix":"// before\nconst cleanId = arxivId.replace('http://arxiv.org/abs/', '').replace('https://arxiv.org/abs/', '')\nconst pdfUrl = `https://arxiv.org/pdf/${cleanId}.pdf`\n// after: also strip abs/ and trailing version noise\nconst cleanId = arxivId.replace(/^https?:\\/\\/arxiv\\.org\\/(abs|pdf)\\//i, '').replace(/\\.pdf$/i, '').replace(/v\\d+$/i, '')\nconst pdfUrl = `https://arxiv.org/pdf/${cleanId}.pdf`","handlingStrategy":"retry","validationCode":"function buildArxivPdfUrl(arxivId: string): string {\n  const cleanId = arxivId.replace(/^https?:\\/\\/arxiv\\.org\\/(abs|pdf)\\//i, '').replace(/\\.pdf$/i, '').replace(/v\\d+$/i, '')\n  return `https://arxiv.org/pdf/${cleanId}.pdf`\n}","typeGuard":null,"tryCatchPattern":"try {\n  return await downloadAndExtractPdf(id)\n} catch (e) {\n  if (/404/.test((e as Error).message)) return '' // withdrawn paper -> skip\n  // retry on 429/5xx, else rethrow\n  throw e\n}","preventionTips":["Normalize the arxiv id (strip URL, .pdf suffix, version) before constructing the PDF URL.","Cache downloaded PDFs by id to avoid re-fetching.","Run PDF downloads serially; they are more aggressively throttled than search."],"tags":["network","arxiv","pdf","http","rate-limit"],"backgroundTag":null,"analyzedSha":"abe4a8601a058047b350c260676826e21dd14101","analyzedAt":"2026-08-12T16:04:40.823Z","schemaVersion":2},"datasetVersion":"2026-08-12T18:17:37.767Z"}