FlowiseAI/Flowise · error · Error

Failed to download PDF: ${response.status} ${response.status

Error message

Failed to download PDF: ${response.status} ${response.statusText}

What it means

Thrown by ArxivTool.downloadAndExtractPdf when the HTTP GET to https://arxiv.org/pdf/<id>.pdf returns a non-2xx status. Status and statusText are interpolated. PDF downloads fail for reasons distinct from the search API — paywalls do not apply, but rate limiting, withdrawn papers, and CDN errors do.

Source

Thrown at packages/components/nodes/tools/Arxiv/core.ts:175

        const response = await fetch(url)
        if (!response.ok) {
            throw new Error(`Arxiv API error: ${response.status} ${response.statusText}`)
        }

        const xmlText = await response.text()
        return this.parseArxivResponse(xmlText)
    }

    private async downloadAndExtractPdf(arxivId: string): Promise<string> {
        // Extract clean arxiv ID from full URL if needed
        const cleanId = arxivId.replace('http://arxiv.org/abs/', '').replace('https://arxiv.org/abs/', '')
        const pdfUrl = `https://arxiv.org/pdf/${cleanId}.pdf`

        this.logger?.info(`[${this.orgId}]: Downloading PDF from: ${pdfUrl}`)

        const response = await fetch(pdfUrl)
        if (!response.ok) {
            throw new Error(`Failed to download PDF: ${response.status} ${response.statusText}`)
        }

        // Get PDF buffer and create blob
        const buffer = await response.buffer()
        const blob = new Blob([new Uint8Array(buffer)])

        // Use PDFLoader to extract text (same as Pdf.ts)
        const loader = new PDFLoader(blob, {
            splitPages: false,
            pdfjs: () =>
                // @ts-ignore
                this.legacyBuild ? import('pdfjs-dist/legacy/build/pdf.js') : import('pdf-parse/lib/pdf.js/v1.10.100/build/pdf.js')
        })

        const docs = await loader.load()
        return docs.map((doc) => doc.pageContent).join('\n')
    }

View on GitHub (pinned to abe4a8601a)

Solutions

1. Retry on 429/5xx with backoff and reduce concurrency.
2. Verify the arxivId is a clean id (no leading URL, correct version format) before constructing pdfUrl.
3. If the paper is withdrawn, treat 404 as a soft skip and fall back to the abstract summary (set continueOnFailure=true).
4. Cache downloaded PDFs to avoid re-fetching across runs.

Example fix

// before
const cleanId = arxivId.replace('http://arxiv.org/abs/', '').replace('https://arxiv.org/abs/', '')
const pdfUrl = `https://arxiv.org/pdf/${cleanId}.pdf`
// after: also strip abs/ and trailing version noise
const cleanId = arxivId.replace(/^https?:\/\/arxiv\.org\/(abs|pdf)\//i, '').replace(/\.pdf$/i, '').replace(/v\d+$/i, '')
const pdfUrl = `https://arxiv.org/pdf/${cleanId}.pdf`

function buildArxivPdfUrl(arxivId: string): string {
  const cleanId = arxivId.replace(/^https?:\/\/arxiv\.org\/(abs|pdf)\//i, '').replace(/\.pdf$/i, '').replace(/v\d+$/i, '')
  return `https://arxiv.org/pdf/${cleanId}.pdf`
}

try {
  return await downloadAndExtractPdf(id)
} catch (e) {
  if (/404/.test((e as Error).message)) return '' // withdrawn paper -> skip
  // retry on 429/5xx, else rethrow
  throw e
}

When it happens

Trigger: Paper was withdrawn and the PDF endpoint returns 404; arxiv.org CDN returns 503 under load; 429 because PDF downloads are more aggressively rate-limited than search; the id contained a version suffix the URL builder did not expect.

Common situations: Bulk ingestion of many papers in a loop; old papers whose PDF was removed; running from a shared cloud IP that Arxiv throttles for PDF downloads.

Related errors

• Arxiv API error: ${response.status} ${response.statusText}
• ${errorMessage}
• HTTP error! status: ${response.status}
• Failed to fetch ${url}: ${error}
• Failed to post ${url}: ${error}

AI-assisted analysis of FlowiseAI/Flowise@abe4a8601a (2026-08-12). Data as JSON: /api/errors/a96412e5678b7eef. Report an issue: GitHub.