What causes "Empty content for ${link}. Skipping." in Mintplex-Labs/anything-llm?

Open the URL in a headless browser yourself to confirm what the DOM looks like without JS-rendered content.

Empty content for ${link}. Skipping.

Error message

Empty content for ${link}. Skipping.

What it means

The WebsiteDepth crawler loaded the page with Puppeteer (waitUntil networkidle2) but document.body.innerText was an empty string, so the link is skipped and never embedded. The page 'loaded' yet produced no extractable text.

Source

Thrown at collector/utils/extensions/WebsiteDepth/index.js:171

    try {
      const loader = new PuppeteerWebBaseLoader(link, {
        launchOptions: {
          headless: launchConfig.headless,
          ignoreHTTPSErrors: true,
          args: runtimeSettings.get("browserLaunchArgs"),
        },
        gotoOptions: { waitUntil: "networkidle2" },
        async evaluate(page, browser) {
          const result = await page.evaluate(() => document.body.innerText);
          await browser.close();
          return result;
        },
      });
      const docs = await loader.load();
      const content = docs[0].pageContent;

      if (!content.length) {
        console.warn(`Empty content for ${link}. Skipping.`);
        continue;
      }

      const url = new URL(link);
      const decodedPathname = decodeURIComponent(url.pathname);
      const filename = `${url.hostname}${decodedPathname.replace(/\//g, "_")}`;

      const data = {
        id: v4(),
        url: "file://" + slugify(filename) + ".html",
        title: slugify(filename) + ".html",
        docAuthor: "no author found",
        description: "No description found.",
        docSource: "URL link uploaded by the user.",
        chunkSource: `link://${link}`,
        published: new Date().toLocaleString(),
        wordCount: content.split(" ").length,
        pageContent: content,

View on GitHub (pinned to 20f6d3546c)

Solutions

Open the URL in a headless browser yourself to confirm what the DOM looks like without JS-rendered content.
Add a wait for a content-bearing selector or extra delay before extracting innerText.
If bot-blocking is the cause, run from an allowed IP/user-agent or source the content another way (API, raw HTML export).
Accept the skip — the crawler intentionally moves to the next link.

Example fix

// before
gotoOptions: { waitUntil: "networkidle2" },

// after
gotoOptions: { waitUntil: "networkidle2", timeout: 60000 },
// and after load:
await page.waitForSelector("main, article, #content", { timeout: 15000 }).catch(() => {});

Defensive patterns

Strategy: fallback

Validate before calling

// Cheap preflight before deep-crawling a link:
const head = await fetch(link);
const type = head.headers.get('content-type') || '';
if (!head.ok || !type.includes('text/html')) skipLink(link);

Prevention

Add waitForSelector for a content container before extracting innerText.
Seed crawls with URLs known to render server-side or expose static HTML.
Run headless checks on target pages before scheduling deep crawls of them.
Treat skips as expected for bot-walled or fully client-rendered pages; source those via APIs instead.

When it happens

Trigger: SPA that renders content via XHR after networkidle fires; bot-detection serving an empty shell; a page that is all canvas/images/video; redirect chains landing on a blank interstitial; pages requiring interaction (cookie wall, login).

Common situations: React/Vue apps with late hydration; Cloudflare or Akamai challenges; links extracted from sitemaps that 200 but render nothing headless.

Related errors

This document has no readable content that could be found.

AI-assisted analysis of Mintplex-Labs/anything-llm@20f6d3546c (2026-08-18). Data as JSON: /api/errors/1a07d498396bf9a2. Report an issue: GitHub.