Mintplex-Labs/anything-llm · error · Error

HTTP ${res.status}: ${res.statusText}

Error message

HTTP ${res.status}: ${res.statusText}

What it means

downloadURIToFile fetches a URL and throws if the response is not ok (res.ok === false). The outer try/catch converts the throw into { success: false, reason } — callers see the reason string, not a thrown exception.

Source

Thrown at collector/utils/downloadURIToFile/index.js:46

 * @returns {Promise<{success: boolean, fileLocation: string|null, reason: string|null}>} - The path to the downloaded file
 */
async function downloadURIToFile(url, maxTimeout = 10_000) {
  if (!url || typeof url !== "string" || !validURL(url))
    return { success: false, reason: "Not a valid URL.", fileLocation: null };

  try {
    const abortController = new AbortController();
    const timeout = setTimeout(() => {
      abortController.abort();
      console.error(
        `Timeout ${maxTimeout}ms reached while downloading file for URL:`,
        url.toString()
      );
    }, maxTimeout);

    const res = await fetch(url, { signal: abortController.signal })
      .then((res) => {
        if (!res.ok) throw new Error(`HTTP ${res.status}: ${res.statusText}`);
        return res;
      })
      .finally(() => clearTimeout(timeout));

    const urlObj = new URL(url);
    const sluggedPath = slugify(urlObj.pathname, { lower: true });
    let filename = `${urlObj.hostname}-${sluggedPath}`;

    const existingExt = path.extname(filename).toLowerCase();
    const { SUPPORTED_FILETYPE_CONVERTERS } = require("../constants");

    // If the filename does not already have a supported file extension,
    // try to infer one from the response Content-Type header.
    // This handles URLs like https://arxiv.org/pdf/2307.10265 where the
    // path has no explicit extension but the server responds with
    // Content-Type: application/pdf.
    if (!SUPPORTED_FILETYPE_CONVERTERS.hasOwnProperty(existingExt)) {
      const { parseContentType } = require("../../processLink/helpers");

View on GitHub (pinned to 526360e320)

Solutions

1. Open the URL in a browser to confirm reachability and the status code.
2. Add authentication headers if the resource requires them.
3. Retry on 5xx/429 with backoff; treat 4xx (except 429) as permanent.
4. Pass a larger maxTimeout for slow sources (default is 10000 ms).

Example fix

// before
if (!res.ok) throw new Error(`HTTP ${res.status}: ${res.statusText}`);

// after — include the URL and mark transient errors retryable
if (!res.ok) {
  const e = new Error(`HTTP ${res.status} ${res.statusText} for ${url}`);
  if (res.status >= 500 || res.status === 429) e.retryable = true;
  throw e;
}

async function reachable(url, { timeout = 10000 } = {}) {
  const ac = new AbortController();
  const t = setTimeout(() => ac.abort(), timeout);
  try {
    const r = await fetch(url, { method: "HEAD", signal: ac.signal });
    return r.ok;
  } catch { return false; }
  finally { clearTimeout(t); }
}

const { success, reason } = await downloadURIToFile(url);
if (!success) {
  if (/HTTP 5\d\d|429/.test(reason)) { /* retry with backoff */ }
  else { /* permanent failure — skip */ }
}

When it happens

Trigger: Any non-2xx HTTP response: 404 (not found), 403/401 (auth/forbidden), 429 (rate limited), 5xx (server error), or fetch rejecting (DNS, TLS, abort on timeout).

Common situations: Link rot (404); paywalled/protected resources (403); rate-limited CDNs (429); transient 5xx; wrong URL pasted; slow server hitting the default 10s timeout.

Related errors

• res.statusText || "Error generating api key."
• res.statusText || "Error finding custom models."
• Failed to fetch edited image: ${imgRes.status}
• Failed to fetch generated image: ${res.status}
• Deepgram transcription failed (${response.status}) - ${errBo

AI-assisted analysis of Mintplex-Labs/anything-llm@526360e320 (2026-08-13). Data as JSON: /api/errors/36e709cadff8752f. Report an issue: GitHub.